[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:11:42.315874 22629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.25.126:45525
I20260709 12:11:42.317462 22629 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:11:42.318562 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:42.334919 22636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:42.334964 22635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:42.334971 22638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:43.460707 22637 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1123 milliseconds
I20260709 12:11:43.460856 22629 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:11:43.462194 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:43.462442 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:43.462651 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599103462631 us; error 0 us; skew 500 ppm
I20260709 12:11:43.463510 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:43.469766 22629 webserver.cc:533] Webserver started at http://127.22.25.126:42429/ using document root <none> and password file <none>
I20260709 12:11:43.470696 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:43.470916 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:43.471390 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:43.477510 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/master-0-root/instance:
uuid: "23d5b23a357149e7991ed3c7f5ce159b"
format_stamp: "Formatted at 2026-07-09 12:11:43 on dist-test-slave-sc22"
I20260709 12:11:43.487004 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:11:43.495857 22644 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:43.497269 22629 fs_manager.cc:730] Time spent opening block manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:11:43.497639 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/master-0-root
uuid: "23d5b23a357149e7991ed3c7f5ce159b"
format_stamp: "Formatted at 2026-07-09 12:11:43 on dist-test-slave-sc22"
I20260709 12:11:43.497974 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:43.544232 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:43.545828 22629 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:11:43.546319 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:43.617372 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.126:45525
I20260709 12:11:43.617437 22695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.126:45525 every 8 connection(s)
I20260709 12:11:43.625591 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:43.648413 22696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: Bootstrap starting.
I20260709 12:11:43.654373 22696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:43.656288 22696 log.cc:826] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: Log is configured to *not* fsync() on all Append() calls
I20260709 12:11:43.661135 22696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: No bootstrap required, opened a new log
I20260709 12:11:43.682471 22696 raft_consensus.cc:359] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER }
I20260709 12:11:43.683055 22696 raft_consensus.cc:385] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:43.683266 22696 raft_consensus.cc:740] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d5b23a357149e7991ed3c7f5ce159b, State: Initialized, Role: FOLLOWER
I20260709 12:11:43.684155 22696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER }
I20260709 12:11:43.684664 22696 raft_consensus.cc:399] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:11:43.684930 22696 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:11:43.685252 22696 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:43.689491 22696 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER }
I20260709 12:11:43.690265 22696 leader_election.cc:304] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23d5b23a357149e7991ed3c7f5ce159b; no voters: 
I20260709 12:11:43.692005 22696 leader_election.cc:290] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:11:43.692297 22699 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:43.694770 22699 raft_consensus.cc:697] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 1 LEADER]: Becoming Leader. State: Replica: 23d5b23a357149e7991ed3c7f5ce159b, State: Running, Role: LEADER
I20260709 12:11:43.695734 22699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER }
I20260709 12:11:43.696802 22696 sys_catalog.cc:565] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:11:43.706737 22701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23d5b23a357149e7991ed3c7f5ce159b. Latest consensus state: current_term: 1 leader_uuid: "23d5b23a357149e7991ed3c7f5ce159b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER } }
I20260709 12:11:43.707360 22700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23d5b23a357149e7991ed3c7f5ce159b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d5b23a357149e7991ed3c7f5ce159b" member_type: VOTER } }
I20260709 12:11:43.707666 22701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:43.708004 22700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:43.710863 22709 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:11:43.724089 22709 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:11:43.726478 22629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:11:43.742802 22709 catalog_manager.cc:1383] Generated new cluster ID: 498580718e594d70ae4a73bd65920edc
I20260709 12:11:43.743147 22709 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:11:43.758896 22709 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:11:43.761184 22709 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:11:43.792718 22709 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: Generated new TSK 0
I20260709 12:11:43.794059 22709 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:11:43.858695 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:43.868577 22718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:43.869694 22719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:43.873564 22721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:43.873867 22629 server_base.cc:1061] running on GCE node
I20260709 12:11:43.874888 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:43.875183 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:43.875389 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599103875371 us; error 0 us; skew 500 ppm
I20260709 12:11:43.876211 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:43.880679 22629 webserver.cc:533] Webserver started at http://127.22.25.65:46665/ using document root <none> and password file <none>
I20260709 12:11:43.881654 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:43.881973 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:43.882368 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:43.884097 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-0-root/instance:
uuid: "57e418e6877547ea92d981055691e9fd"
format_stamp: "Formatted at 2026-07-09 12:11:43 on dist-test-slave-sc22"
I20260709 12:11:43.891461 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:11:43.897018 22726 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:43.897889 22629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.003s
I20260709 12:11:43.898198 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
uuid: "57e418e6877547ea92d981055691e9fd"
format_stamp: "Formatted at 2026-07-09 12:11:43 on dist-test-slave-sc22"
I20260709 12:11:43.898516 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:43.930397 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:43.931973 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:43.934628 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:43.939311 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:43.939554 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:43.939853 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:43.940021 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.034611 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.65:45543
I20260709 12:11:44.034698 22788 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.65:45543 every 8 connection(s)
I20260709 12:11:44.041903 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:44.056396 22793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:44.064762 22794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:44.068099 22796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:44.068673 22629 server_base.cc:1061] running on GCE node
I20260709 12:11:44.069509 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:44.069731 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:44.070127 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599104070090 us; error 0 us; skew 500 ppm
I20260709 12:11:44.071190 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:44.074967 22789 heartbeater.cc:344] Connected to a master server at 127.22.25.126:45525
I20260709 12:11:44.074993 22629 webserver.cc:533] Webserver started at http://127.22.25.66:43087/ using document root <none> and password file <none>
I20260709 12:11:44.075683 22789 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:44.075892 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:44.076174 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:44.076534 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:44.076961 22789 heartbeater.cc:507] Master 127.22.25.126:45525 requested a full tablet report, sending...
I20260709 12:11:44.078158 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-1-root/instance:
uuid: "6a94dc37206a4330b414b37ea06b40be"
format_stamp: "Formatted at 2026-07-09 12:11:44 on dist-test-slave-sc22"
I20260709 12:11:44.080919 22661 ts_manager.cc:194] Registered new tserver with Master: 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543)
I20260709 12:11:44.084159 22661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41214
I20260709 12:11:44.084975 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.009s	sys 0.000s
I20260709 12:11:44.090576 22801 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.091691 22629 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:11:44.092058 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
uuid: "6a94dc37206a4330b414b37ea06b40be"
format_stamp: "Formatted at 2026-07-09 12:11:44 on dist-test-slave-sc22"
I20260709 12:11:44.092433 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:44.111253 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:44.113008 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:44.115536 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:44.119239 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:44.119517 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.119834 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:44.120049 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.206799 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.66:44929
I20260709 12:11:44.207191 22863 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.66:44929 every 8 connection(s)
I20260709 12:11:44.212378 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:44.220924 22867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:44.223538 22868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:44.226800 22864 heartbeater.cc:344] Connected to a master server at 127.22.25.126:45525
I20260709 12:11:44.227254 22864 heartbeater.cc:461] Registering TS with master...
W20260709 12:11:44.228077 22870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:44.228374 22864 heartbeater.cc:507] Master 127.22.25.126:45525 requested a full tablet report, sending...
I20260709 12:11:44.228755 22629 server_base.cc:1061] running on GCE node
I20260709 12:11:44.229722 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:44.229985 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:44.230158 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599104230141 us; error 0 us; skew 500 ppm
I20260709 12:11:44.230865 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:44.231019 22661 ts_manager.cc:194] Registered new tserver with Master: 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:44.232867 22661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41224
I20260709 12:11:44.234197 22629 webserver.cc:533] Webserver started at http://127.22.25.67:42371/ using document root <none> and password file <none>
I20260709 12:11:44.234812 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:44.235078 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:44.235394 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:44.236554 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-2-root/instance:
uuid: "ed03529aa1c04c2d9d23061fba504067"
format_stamp: "Formatted at 2026-07-09 12:11:44 on dist-test-slave-sc22"
I20260709 12:11:44.241266 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:11:44.244939 22875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.245769 22629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:11:44.246066 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
uuid: "ed03529aa1c04c2d9d23061fba504067"
format_stamp: "Formatted at 2026-07-09 12:11:44 on dist-test-slave-sc22"
I20260709 12:11:44.246364 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599102143852-22629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:44.261464 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:44.262713 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:44.264870 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:44.268188 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:44.268404 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.268642 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:44.268800 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:44.358074 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.67:43431
I20260709 12:11:44.358224 22937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.67:43431 every 8 connection(s)
I20260709 12:11:44.373850 22938 heartbeater.cc:344] Connected to a master server at 127.22.25.126:45525
I20260709 12:11:44.374208 22938 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:44.374967 22938 heartbeater.cc:507] Master 127.22.25.126:45525 requested a full tablet report, sending...
I20260709 12:11:44.376847 22661 ts_manager.cc:194] Registered new tserver with Master: ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:44.377960 22629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013427132s
I20260709 12:11:44.378131 22661 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41238
I20260709 12:11:44.433564 22661 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41254:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:11:44.437273 22661 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:11:44.529824 22903 tablet_service.cc:1511] Processing CreateTablet for tablet d5b2635f595b4b60a95dbbfc8967a5f8 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:44.530686 22902 tablet_service.cc:1511] Processing CreateTablet for tablet 6a96aa0ed6654a62a92c20a4b68a530a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:44.532976 22900 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8ccc590e3e4beaad73bd86c68df22a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:44.535176 22901 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb3e39dba904fc997edf2087a214803 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:44.536597 22903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b2635f595b4b60a95dbbfc8967a5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.546494 22901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb3e39dba904fc997edf2087a214803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.537968 22753 tablet_service.cc:1511] Processing CreateTablet for tablet 6a96aa0ed6654a62a92c20a4b68a530a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:44.544785 22750 tablet_service.cc:1511] Processing CreateTablet for tablet 06bfc2e074c54c1a8b464c942281ed08 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:44.551103 22754 tablet_service.cc:1511] Processing CreateTablet for tablet d5b2635f595b4b60a95dbbfc8967a5f8 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:44.556140 22900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8ccc590e3e4beaad73bd86c68df22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.560411 22751 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8ccc590e3e4beaad73bd86c68df22a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:44.559875 22825 tablet_service.cc:1511] Processing CreateTablet for tablet 06bfc2e074c54c1a8b464c942281ed08 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:44.563858 22753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a96aa0ed6654a62a92c20a4b68a530a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.562572 22828 tablet_service.cc:1511] Processing CreateTablet for tablet 6a96aa0ed6654a62a92c20a4b68a530a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:44.562484 22829 tablet_service.cc:1511] Processing CreateTablet for tablet d5b2635f595b4b60a95dbbfc8967a5f8 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:44.559899 22826 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8ccc590e3e4beaad73bd86c68df22a (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:44.567553 22825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06bfc2e074c54c1a8b464c942281ed08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.566484 22899 tablet_service.cc:1511] Processing CreateTablet for tablet 06bfc2e074c54c1a8b464c942281ed08 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:44.571852 22902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a96aa0ed6654a62a92c20a4b68a530a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.571936 22751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8ccc590e3e4beaad73bd86c68df22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.569731 22829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b2635f595b4b60a95dbbfc8967a5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.569715 22749 tablet_service.cc:1511] Processing CreateTablet for tablet 27cea1a6ad774898ba30a64d0da5987f (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:44.566390 22827 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb3e39dba904fc997edf2087a214803 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:44.576618 22752 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb3e39dba904fc997edf2087a214803 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:44.577011 22828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a96aa0ed6654a62a92c20a4b68a530a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.580744 22754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b2635f595b4b60a95dbbfc8967a5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.583838 22898 tablet_service.cc:1511] Processing CreateTablet for tablet 27cea1a6ad774898ba30a64d0da5987f (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:44.586834 22899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06bfc2e074c54c1a8b464c942281ed08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.577374 22824 tablet_service.cc:1511] Processing CreateTablet for tablet 27cea1a6ad774898ba30a64d0da5987f (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:44.592993 22748 tablet_service.cc:1511] Processing CreateTablet for tablet 840f0061cce4401796a5422bed9ddcef (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:44.592126 22826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8ccc590e3e4beaad73bd86c68df22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.592648 22898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cea1a6ad774898ba30a64d0da5987f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.601140 22824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cea1a6ad774898ba30a64d0da5987f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.619446 22750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06bfc2e074c54c1a8b464c942281ed08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.628002 22748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840f0061cce4401796a5422bed9ddcef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.630973 22749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cea1a6ad774898ba30a64d0da5987f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.637105 22752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb3e39dba904fc997edf2087a214803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.620255 22827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb3e39dba904fc997edf2087a214803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.653460 22823 tablet_service.cc:1511] Processing CreateTablet for tablet 840f0061cce4401796a5422bed9ddcef (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:44.660168 22823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840f0061cce4401796a5422bed9ddcef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.696522 22897 tablet_service.cc:1511] Processing CreateTablet for tablet 840f0061cce4401796a5422bed9ddcef (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:44.702229 22897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840f0061cce4401796a5422bed9ddcef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:44.733676 22708 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:44.737071 22822 tablet_service.cc:1511] Processing CreateTablet for tablet f02ff32889134e3c97a80a38a5d70b01 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:44.741114 22822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02ff32889134e3c97a80a38a5d70b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.745445 22896 tablet_service.cc:1511] Processing CreateTablet for tablet f02ff32889134e3c97a80a38a5d70b01 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:44.728045 22747 tablet_service.cc:1511] Processing CreateTablet for tablet f02ff32889134e3c97a80a38a5d70b01 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:44.743337 22746 tablet_service.cc:1511] Processing CreateTablet for tablet 54ba99eba7854fb59ff6d0c3e4388d94 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:44.753814 22821 tablet_service.cc:1511] Processing CreateTablet for tablet 54ba99eba7854fb59ff6d0c3e4388d94 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:44.754021 22896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02ff32889134e3c97a80a38a5d70b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.756871 22746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ba99eba7854fb59ff6d0c3e4388d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.760185 22821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ba99eba7854fb59ff6d0c3e4388d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.760298 22747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02ff32889134e3c97a80a38a5d70b01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.790499 22895 tablet_service.cc:1511] Processing CreateTablet for tablet 54ba99eba7854fb59ff6d0c3e4388d94 (DEFAULT_TABLE table=test-workload [id=f0352fa2438b496891bc06dafdb952f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:44.793716 22895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ba99eba7854fb59ff6d0c3e4388d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:44.876957 22956 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:44.878643 22957 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:44.908785 22958 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:44.919909 22956 tablet_bootstrap.cc:654] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.920557 22957 tablet_bootstrap.cc:654] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.922358 22958 tablet_bootstrap.cc:654] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.934223 22958 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:44.934566 22957 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:44.934749 22958 ts_tablet_manager.cc:1403] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.006s
I20260709 12:11:44.935230 22957 ts_tablet_manager.cc:1403] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.005s
I20260709 12:11:44.937947 22958 raft_consensus.cc:359] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.938571 22956 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:44.938134 22957 raft_consensus.cc:359] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.938843 22958 raft_consensus.cc:385] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:44.939083 22956 ts_tablet_manager.cc:1403] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.017s
I20260709 12:11:44.939083 22957 raft_consensus.cc:385] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:44.939210 22958 raft_consensus.cc:740] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:44.939805 22957 raft_consensus.cc:740] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:44.940269 22958 consensus_queue.cc:260] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.940739 22957 consensus_queue.cc:260] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.943084 22956 raft_consensus.cc:359] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.943923 22956 raft_consensus.cc:385] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:44.944259 22956 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:44.945086 22956 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.950270 22864 heartbeater.cc:499] Master 127.22.25.126:45525 was elected leader, sending a full tablet report...
I20260709 12:11:44.951507 22957 ts_tablet_manager.cc:1434] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 12:11:44.952858 22957 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:44.959944 22789 heartbeater.cc:499] Master 127.22.25.126:45525 was elected leader, sending a full tablet report...
I20260709 12:11:44.960160 22957 tablet_bootstrap.cc:654] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.968674 22938 heartbeater.cc:499] Master 127.22.25.126:45525 was elected leader, sending a full tablet report...
I20260709 12:11:44.969676 22956 ts_tablet_manager.cc:1434] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.030s	user 0.019s	sys 0.004s
I20260709 12:11:44.970736 22956 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:44.971968 22958 ts_tablet_manager.cc:1434] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.037s	user 0.016s	sys 0.000s
I20260709 12:11:44.972887 22957 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:44.973079 22958 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:44.973395 22957 ts_tablet_manager.cc:1403] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
W20260709 12:11:44.974992 22865 tablet.cc:2406] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:11:44.976799 22957 raft_consensus.cc:359] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:44.977553 22957 raft_consensus.cc:385] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:44.977887 22957 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:44.978150 22956 tablet_bootstrap.cc:654] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.978645 22957 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:44.981192 22957 ts_tablet_manager.cc:1434] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:11:44.982275 22957 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:44.989499 22957 tablet_bootstrap.cc:654] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.990350 22956 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:44.990875 22956 ts_tablet_manager.cc:1403] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 12:11:44.993746 22956 raft_consensus.cc:359] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.995215 22956 raft_consensus.cc:385] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:44.995539 22956 raft_consensus.cc:740] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:44.996289 22956 consensus_queue.cc:260] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:44.998270 22958 tablet_bootstrap.cc:654] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:44.998471 22956 ts_tablet_manager.cc:1434] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:11:45.000118 22956 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.006912 22956 tablet_bootstrap.cc:654] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.014132 22957 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.014846 22957 ts_tablet_manager.cc:1403] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.002s
I20260709 12:11:45.020038 22957 raft_consensus.cc:359] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.021030 22957 raft_consensus.cc:385] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.021442 22957 raft_consensus.cc:740] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.022508 22957 consensus_queue.cc:260] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.025163 22957 ts_tablet_manager.cc:1434] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:11:45.033671 22957 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:45.034500 22958 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.035045 22958 ts_tablet_manager.cc:1403] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.062s	user 0.031s	sys 0.005s
I20260709 12:11:45.038084 22958 raft_consensus.cc:359] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.038887 22958 raft_consensus.cc:385] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.039224 22958 raft_consensus.cc:740] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.040865 22958 consensus_queue.cc:260] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.043159 22957 tablet_bootstrap.cc:654] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.043944 22956 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.044450 22956 ts_tablet_manager.cc:1403] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.025s
I20260709 12:11:45.045109 22958 ts_tablet_manager.cc:1434] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:11:45.046226 22958 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.053650 22958 tablet_bootstrap.cc:654] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.056563 22956 raft_consensus.cc:359] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.057361 22956 raft_consensus.cc:385] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.057677 22956 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.058418 22956 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.059731 22957 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.060230 22957 ts_tablet_manager.cc:1403] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.027s	user 0.018s	sys 0.004s
I20260709 12:11:45.060324 22956 ts_tablet_manager.cc:1434] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:11:45.061311 22956 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.061937 22958 tablet_bootstrap.cc:492] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.062414 22958 ts_tablet_manager.cc:1403] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:11:45.063201 22957 raft_consensus.cc:359] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.064019 22957 raft_consensus.cc:385] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.064335 22957 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.065203 22957 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.067118 22957 ts_tablet_manager.cc:1434] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:11:45.068058 22957 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:45.065353 22958 raft_consensus.cc:359] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.071178 22958 raft_consensus.cc:385] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.071542 22958 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.071743 22962 raft_consensus.cc:493] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.072186 22962 raft_consensus.cc:515] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.072387 22958 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.075423 22962 leader_election.cc:290] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.077977 22957 tablet_bootstrap.cc:654] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.081722 22958 ts_tablet_manager.cc:1434] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:11:45.082633 22958 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.083639 22956 tablet_bootstrap.cc:654] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.091123 22958 tablet_bootstrap.cc:654] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.105782 22963 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.106411 22963 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.110553 22963 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.111089 22963 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.111567 22963 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.111939 22972 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.112519 22972 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.113716 22963 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543)
I20260709 12:11:45.114677 22972 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.144320 22972 raft_consensus.cc:493] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.144868 22972 raft_consensus.cc:515] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.147058 22972 leader_election.cc:290] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.150652 22957 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.151216 22957 ts_tablet_manager.cc:1403] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.040s
I20260709 12:11:45.154511 22957 raft_consensus.cc:359] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.155309 22957 raft_consensus.cc:385] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.155678 22957 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.156446 22957 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.169097 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.170044 22839 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.171835 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.172606 22839 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.174332 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.174510 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.175235 22763 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 0.
I20260709 12:11:45.176018 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.176749 22839 raft_consensus.cc:2468] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
W20260709 12:11:45.176836 22803 leader_election.cc:343] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.177982 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.178705 22912 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 0.
I20260709 12:11:45.178972 22803 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.180222 22982 raft_consensus.cc:2804] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.180683 22982 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.181070 22982 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.177665 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
W20260709 12:11:45.189174 22802 leader_election.cc:343] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.189682 22802 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be; no voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.190526 22963 raft_consensus.cc:2749] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.190858 22982 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.194553 22730 leader_election.cc:304] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.194799 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:45.195626 22972 raft_consensus.cc:2804] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.196017 22972 raft_consensus.cc:493] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.196051 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.196460 22972 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.196666 22913 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.196370 22730 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.198055 22730 leader_election.cc:304] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.195475 22763 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.205570 22985 raft_consensus.cc:2804] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.206867 22985 raft_consensus.cc:493] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.207378 22985 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.209533 22763 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 1.
I20260709 12:11:45.208141 22972 raft_consensus.cc:515] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.211838 22962 raft_consensus.cc:2764] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:11:45.212638 22803 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.213613 22963 raft_consensus.cc:2804] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.215701 22972 leader_election.cc:290] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.216153 22958 tablet_bootstrap.cc:492] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.216593 22958 ts_tablet_manager.cc:1403] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.134s	user 0.030s	sys 0.032s
I20260709 12:11:45.216755 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.217396 22839 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.209127 22957 ts_tablet_manager.cc:1434] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.004s
I20260709 12:11:45.219465 22957 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:45.220289 22912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.221340 22956 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.221827 22956 ts_tablet_manager.cc:1403] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.161s	user 0.039s	sys 0.045s
I20260709 12:11:45.221710 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.223515 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.226018 22957 tablet_bootstrap.cc:654] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:45.226229 22727 leader_election.cc:343] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:45.227754 22727 leader_election.cc:343] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.228873 22956 raft_consensus.cc:359] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.229635 22956 raft_consensus.cc:385] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.229918 22956 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.224107 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.206372 22982 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Requested vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543)
I20260709 12:11:45.211136 22913 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 1.
I20260709 12:11:45.230650 22956 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
W20260709 12:11:45.236315 22727 leader_election.cc:343] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.237413 22839 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.240091 22730 leader_election.cc:304] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.240871 22963 raft_consensus.cc:697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:45.241220 22972 raft_consensus.cc:2804] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.243678 22972 raft_consensus.cc:697] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:45.245306 22972 consensus_queue.cc:237] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.244380 22963 consensus_queue.cc:237] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:45.250452 22957 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.246405 22958 raft_consensus.cc:359] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.250836 22958 raft_consensus.cc:385] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.250931 22957 ts_tablet_manager.cc:1403] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.012s
I20260709 12:11:45.245699 22956 ts_tablet_manager.cc:1434] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:11:45.251196 22958 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.252815 22985 raft_consensus.cc:515] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.255125 22958 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.255650 22985 leader_election.cc:290] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.256392 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.256958 22956 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.257017 22839 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.257566 22958 ts_tablet_manager.cc:1434] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:11:45.258646 22958 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.262518 22964 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.263092 22964 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.263794 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
W20260709 12:11:45.267326 22727 leader_election.cc:343] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.267917 22964 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.271067 22957 raft_consensus.cc:359] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.274298 22957 raft_consensus.cc:385] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.274649 22957 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.274549 22839 raft_consensus.cc:2468] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.278663 22730 leader_election.cc:304] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.283844 22972 raft_consensus.cc:2804] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.284390 22972 raft_consensus.cc:697] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:45.283985 22957 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.285197 22972 consensus_queue.cc:237] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.287030 22957 ts_tablet_manager.cc:1434] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.003s
I20260709 12:11:45.288111 22957 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:45.277097 22986 raft_consensus.cc:493] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.289206 22986 raft_consensus.cc:515] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.286499 22659 catalog_manager.cc:5697] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 1 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.297444 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.295861 22986 leader_election.cc:290] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.299643 22763 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 0.
I20260709 12:11:45.301797 22803 leader_election.cc:304] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.301831 22956 tablet_bootstrap.cc:654] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.303107 22986 raft_consensus.cc:2804] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.303740 22986 raft_consensus.cc:493] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.311992 22986 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.325891 22661 catalog_manager.cc:5697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 1 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.329454 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.330268 22913 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 0.
I20260709 12:11:45.332715 22958 tablet_bootstrap.cc:654] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.349382 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.350646 22986 raft_consensus.cc:515] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
W20260709 12:11:45.351199 22877 leader_election.cc:343] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.353722 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:45.354311 22763 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.356571 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.357196 22913 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.358307 22956 tablet_bootstrap.cc:492] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.358781 22956 ts_tablet_manager.cc:1403] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.102s	user 0.009s	sys 0.010s
I20260709 12:11:45.360224 22763 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 1.
I20260709 12:11:45.361419 22803 leader_election.cc:304] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.362179 22963 raft_consensus.cc:2804] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.362628 22963 raft_consensus.cc:697] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:45.363302 22963 consensus_queue.cc:237] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.367517 22956 raft_consensus.cc:359] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.368358 22956 raft_consensus.cc:385] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.368681 22956 raft_consensus.cc:740] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.369465 22956 consensus_queue.cc:260] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.371204 22956 ts_tablet_manager.cc:1434] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:11:45.371846 22957 tablet_bootstrap.cc:654] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.372606 22956 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.378839 22913 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 1.
I20260709 12:11:45.379081 22956 tablet_bootstrap.cc:654] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.384729 22957 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.385437 22957 ts_tablet_manager.cc:1403] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.098s	user 0.004s	sys 0.014s
I20260709 12:11:45.388091 22986 leader_election.cc:290] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.389631 22957 raft_consensus.cc:359] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.390411 22964 raft_consensus.cc:493] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.390751 22957 raft_consensus.cc:385] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.391108 22957 raft_consensus.cc:740] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.390935 22964 raft_consensus.cc:515] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.391965 22957 consensus_queue.cc:260] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.393280 22964 leader_election.cc:290] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.394919 22957 ts_tablet_manager.cc:1434] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:11:45.395164 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.395381 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.396013 22957 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
I20260709 12:11:45.395968 22839 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 0.
I20260709 12:11:45.396234 22958 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.396740 22958 ts_tablet_manager.cc:1403] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.138s	user 0.005s	sys 0.010s
I20260709 12:11:45.396615 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.397547 22879 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 57e418e6877547ea92d981055691e9fd
W20260709 12:11:45.398406 22877 leader_election.cc:343] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.398669 22964 raft_consensus.cc:2804] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.399025 22964 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.399356 22964 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:11:45.399641 22879 leader_election.cc:343] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.400085 22879 leader_election.cc:304] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed03529aa1c04c2d9d23061fba504067; no voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:45.403813 22957 tablet_bootstrap.cc:654] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.400758 22958 raft_consensus.cc:359] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.406296 22958 raft_consensus.cc:385] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.406643 22958 raft_consensus.cc:740] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.407454 22958 consensus_queue.cc:260] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.410178 22958 ts_tablet_manager.cc:1434] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:11:45.411377 22958 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.412591 22986 raft_consensus.cc:493] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.413120 22986 raft_consensus.cc:515] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.413626 22964 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.415448 22986 leader_election.cc:290] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.415733 23002 raft_consensus.cc:2749] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.416326 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.416988 22763 raft_consensus.cc:2468] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 0.
I20260709 12:11:45.418301 22803 leader_election.cc:304] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.419203 22958 tablet_bootstrap.cc:654] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.419130 22986 raft_consensus.cc:2804] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.419625 22986 raft_consensus.cc:493] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.419986 22986 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.419346 22661 catalog_manager.cc:5697] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 1 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.420467 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
W20260709 12:11:45.422010 22802 leader_election.cc:343] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.422259 22957 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.422757 22957 ts_tablet_manager.cc:1403] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.027s	user 0.001s	sys 0.017s
I20260709 12:11:45.425464 22957 raft_consensus.cc:359] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.426141 22957 raft_consensus.cc:385] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.426410 22957 raft_consensus.cc:740] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.426625 22986 raft_consensus.cc:515] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.427106 22957 consensus_queue.cc:260] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.428740 22986 leader_election.cc:290] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.429111 22957 ts_tablet_manager.cc:1434] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:11:45.429857 22964 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.429927 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:45.430562 22763 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.431109 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.432363 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd"
W20260709 12:11:45.434235 22877 leader_election.cc:343] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.429942 22957 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: Bootstrap starting.
W20260709 12:11:45.435935 22802 leader_election.cc:343] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.436282 22956 tablet_bootstrap.cc:492] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.436743 22956 ts_tablet_manager.cc:1403] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.011s
I20260709 12:11:45.437268 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.437877 22838 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.439580 22659 catalog_manager.cc:5697] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 1 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.440122 22956 raft_consensus.cc:359] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.440647 22957 tablet_bootstrap.cc:654] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.440838 22956 raft_consensus.cc:385] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.441161 22956 raft_consensus.cc:740] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.442044 22956 consensus_queue.cc:260] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.448807 22763 raft_consensus.cc:2468] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 1.
I20260709 12:11:45.450145 22956 ts_tablet_manager.cc:1434] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 12:11:45.449998 22803 leader_election.cc:304] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.451134 22956 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.452131 22838 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 1.
I20260709 12:11:45.466320 22958 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.468225 22958 ts_tablet_manager.cc:1403] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.057s	user 0.003s	sys 0.026s
I20260709 12:11:45.468854 22879 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 57e418e6877547ea92d981055691e9fd
I20260709 12:11:45.469666 22964 raft_consensus.cc:2804] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.470155 22964 raft_consensus.cc:697] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:45.471071 22958 raft_consensus.cc:359] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.470966 22964 consensus_queue.cc:237] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.471683 22958 raft_consensus.cc:385] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.472023 22958 raft_consensus.cc:740] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.472720 22958 consensus_queue.cc:260] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.474785 22958 ts_tablet_manager.cc:1434] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:11:45.466753 22956 tablet_bootstrap.cc:654] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.478957 22958 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.484004 22957 tablet_bootstrap.cc:492] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: No bootstrap required, opened a new log
I20260709 12:11:45.484512 22957 ts_tablet_manager.cc:1403] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.005s
I20260709 12:11:45.488382 22958 tablet_bootstrap.cc:654] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.488684 22963 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.489238 22963 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.491441 22963 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.493283 22956 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.493286 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.494041 22956 ts_tablet_manager.cc:1403] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.006s
I20260709 12:11:45.493904 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
W20260709 12:11:45.494971 22803 leader_election.cc:343] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:45.495621 22802 leader_election.cc:343] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.496032 22802 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be; no voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.496745 22963 raft_consensus.cc:2749] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.496573 22957 raft_consensus.cc:359] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.497366 22956 raft_consensus.cc:359] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.498075 22957 raft_consensus.cc:385] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.498175 22956 raft_consensus.cc:385] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.498375 22957 raft_consensus.cc:740] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.498518 22956 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.499184 22957 consensus_queue.cc:260] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.499423 22956 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.501705 22957 ts_tablet_manager.cc:1434] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:11:45.502123 22956 ts_tablet_manager.cc:1434] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:11:45.503120 22956 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.506289 22958 tablet_bootstrap.cc:492] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.506912 22958 ts_tablet_manager.cc:1403] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.031s	user 0.005s	sys 0.007s
I20260709 12:11:45.509665 22956 tablet_bootstrap.cc:654] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.510012 22958 raft_consensus.cc:359] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.510807 22958 raft_consensus.cc:385] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.511209 22958 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.512082 22958 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.514708 22958 ts_tablet_manager.cc:1434] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:11:45.515829 22958 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.516934 22658 catalog_manager.cc:5697] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 1 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:45.518615 22963 raft_consensus.cc:493] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.519089 22963 raft_consensus.cc:515] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.521318 22963 leader_election.cc:290] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.522610 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
W20260709 12:11:45.524292 22802 leader_election.cc:343] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.525768 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
W20260709 12:11:45.528884 22803 leader_election.cc:343] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:45.529367 22803 leader_election.cc:304] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be; no voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.530987 22963 raft_consensus.cc:2749] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.536072 22958 tablet_bootstrap.cc:654] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.467563 22986 raft_consensus.cc:2804] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.539510 22986 raft_consensus.cc:697] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:45.540486 22986 consensus_queue.cc:237] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.550086 22958 tablet_bootstrap.cc:492] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.550611 22956 tablet_bootstrap.cc:492] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.550751 22958 ts_tablet_manager.cc:1403] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.004s
I20260709 12:11:45.551296 22956 ts_tablet_manager.cc:1403] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.016s
I20260709 12:11:45.552325 22657 catalog_manager.cc:5697] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 1 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.553510 22958 raft_consensus.cc:359] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.554136 22958 raft_consensus.cc:385] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.554108 22956 raft_consensus.cc:359] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.554548 22958 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.554875 22956 raft_consensus.cc:385] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.555428 22956 raft_consensus.cc:740] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.555289 22958 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.556540 22956 consensus_queue.cc:260] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.557812 22958 ts_tablet_manager.cc:1434] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:11:45.559226 22958 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: Bootstrap starting.
I20260709 12:11:45.559777 22956 ts_tablet_manager.cc:1434] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:11:45.560668 22956 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: Bootstrap starting.
I20260709 12:11:45.564966 22958 tablet_bootstrap.cc:654] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.566293 22956 tablet_bootstrap.cc:654] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.577018 22958 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: No bootstrap required, opened a new log
I20260709 12:11:45.577486 22958 ts_tablet_manager.cc:1403] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:11:45.577893 22956 tablet_bootstrap.cc:492] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: No bootstrap required, opened a new log
I20260709 12:11:45.578379 22956 ts_tablet_manager.cc:1403] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.012s
I20260709 12:11:45.580165 22958 raft_consensus.cc:359] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.580847 22958 raft_consensus.cc:385] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.581161 22958 raft_consensus.cc:740] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.580919 22956 raft_consensus.cc:359] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.581749 22956 raft_consensus.cc:385] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.582095 22956 raft_consensus.cc:740] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.581851 22958 consensus_queue.cc:260] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.583374 22956 consensus_queue.cc:260] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.584641 22958 ts_tablet_manager.cc:1434] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:11:45.585546 22956 ts_tablet_manager.cc:1434] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:11:45.643267 22972 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.643826 22972 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.644387 22913 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.646000 22972 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.646837 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.647740 22838 raft_consensus.cc:2393] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 57e418e6877547ea92d981055691e9fd in current term 1: Already voted for candidate ed03529aa1c04c2d9d23061fba504067 in this term.
I20260709 12:11:45.647656 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.649411 22727 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 57e418e6877547ea92d981055691e9fd; no voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:45.650269 22962 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.647678 22972 raft_consensus.cc:493] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.650961 22972 raft_consensus.cc:515] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.653139 22972 leader_election.cc:290] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.654211 22972 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.654811 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.655619 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.656260 22838 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.657554 22730 leader_election.cc:304] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.658748 22962 raft_consensus.cc:2749] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.659475 22962 raft_consensus.cc:2804] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.659884 22962 raft_consensus.cc:493] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.660257 22962 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.655429 22913 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.667456 22962 raft_consensus.cc:515] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.670823 22962 leader_election.cc:290] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.671577 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.672043 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.672227 22838 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.684618 22913 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.686372 22964 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.686877 22964 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.690474 22964 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.691756 22838 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.693008 22730 leader_election.cc:304] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.693817 22962 raft_consensus.cc:2804] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.692109 22985 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:11:45.694306 22962 raft_consensus.cc:697] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:45.695135 22962 consensus_queue.cc:237] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.696077 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.696714 22839 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 0.
I20260709 12:11:45.698410 22879 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:45.699195 22964 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.699555 22964 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.699504 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.700038 22964 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.700228 22763 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 0.
I20260709 12:11:45.706677 22964 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.707304 22661 catalog_manager.cc:5697] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 1 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.709200 22964 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.710079 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:45.710713 22763 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.712512 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.713119 22839 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.717568 22763 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 1.
I20260709 12:11:45.718923 22877 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:45.719995 22839 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 1.
I20260709 12:11:45.720280 22964 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.721722 22964 raft_consensus.cc:697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:45.721875 22913 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.722599 22964 consensus_queue.cc:237] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.732133 22658 catalog_manager.cc:5697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 1 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:45.737563 22708 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:45.753218 22986 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.765246 22963 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.779851 22962 consensus_queue.cc:1048] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.792884 22913 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.802909 22972 consensus_queue.cc:1048] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.822906 22972 raft_consensus.cc:493] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.823423 22972 raft_consensus.cc:515] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.825636 22972 leader_election.cc:290] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.826090 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.826803 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067" is_pre_election: true
I20260709 12:11:45.826994 22838 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.827344 22913 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 0.
I20260709 12:11:45.828349 22730 leader_election.cc:304] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.829417 22972 raft_consensus.cc:2804] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:45.829824 22972 raft_consensus.cc:493] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.830194 22972 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.835985 22972 raft_consensus.cc:515] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.837978 22972 leader_election.cc:290] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:45.838896 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:45.838981 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:45.839414 22838 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.839625 22913 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.844064 22838 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.845069 22730 leader_election.cc:304] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:45.845713 22972 raft_consensus.cc:2804] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.846103 22972 raft_consensus.cc:697] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:45.846441 22913 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 1.
I20260709 12:11:45.846827 22972 consensus_queue.cc:237] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.850929 22964 raft_consensus.cc:493] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.851410 22964 raft_consensus.cc:515] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:45.853315 22964 leader_election.cc:290] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:45.854411 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a94dc37206a4330b414b37ea06b40be" is_pre_election: true
I20260709 12:11:45.854784 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e418e6877547ea92d981055691e9fd" is_pre_election: true
I20260709 12:11:45.855509 22763 raft_consensus.cc:2393] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed03529aa1c04c2d9d23061fba504067 in current term 1: Already voted for candidate 6a94dc37206a4330b414b37ea06b40be in this term.
I20260709 12:11:45.855791 22661 catalog_manager.cc:5697] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 1 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:45.856475 22877 leader_election.cc:304] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed03529aa1c04c2d9d23061fba504067; no voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:45.857419 22964 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.863055 22964 raft_consensus.cc:2749] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:45.872642 22964 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.876786 22963 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.881697 22963 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.898689 22964 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:11:45.949910 22963 consensus_queue.cc:1048] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.962095 22982 consensus_queue.cc:1048] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:45.999843 22838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:48858
I20260709 12:11:46.000411 22838 raft_consensus.cc:606] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:11:46.014346 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:51904
I20260709 12:11:46.014853 22913 raft_consensus.cc:606] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:11:46.022682 22838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:48858
I20260709 12:11:46.023208 22838 raft_consensus.cc:606] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:11:46.028321 22913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:51904
I20260709 12:11:46.028826 22913 raft_consensus.cc:606] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:46.042287 22764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:46.042783 22764 raft_consensus.cc:606] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:46.048418 22764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:46.048975 22764 raft_consensus.cc:606] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:46.053973 22764 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:46.054489 22764 raft_consensus.cc:606] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:46.059785 22838 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:48858
I20260709 12:11:46.060282 22838 raft_consensus.cc:606] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Received request to transfer leadership to TS ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:46.176342 22972 consensus_queue.cc:1048] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:46.186378 22962 consensus_queue.cc:1048] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:46.196913 22962 raft_consensus.cc:993] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.197417 23024 raft_consensus.cc:1081] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.199313 22838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:46.199827 22838 raft_consensus.cc:493] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.200198 22838 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.205107 22838 raft_consensus.cc:515] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.207670 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:46.208016 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02ff32889134e3c97a80a38a5d70b01" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.207648 22838 leader_election.cc:290] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.208304 22764 raft_consensus.cc:3055] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.208971 22764 raft_consensus.cc:740] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.208709 22913 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.210417 22764 consensus_queue.cc:260] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.212203 22764 raft_consensus.cc:3060] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.216549 22913 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.217945 22802 leader_election.cc:304] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.218797 22982 raft_consensus.cc:2804] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.218926 22764 raft_consensus.cc:2468] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.219313 22982 raft_consensus.cc:697] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.220192 22982 consensus_queue.cc:237] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.227972 22658 catalog_manager.cc:5697] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 1 to 2, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 2 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.269583 22985 raft_consensus.cc:993] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.270059 22962 raft_consensus.cc:1081] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.271926 22839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:46.272501 22839 raft_consensus.cc:493] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.272756 22962 raft_consensus.cc:993] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.272910 22839 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.273216 22962 raft_consensus.cc:1081] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.274910 22838 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:46.275499 22838 raft_consensus.cc:493] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.275898 22838 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.279359 22839 raft_consensus.cc:515] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.282328 22838 raft_consensus.cc:515] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.282586 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:46.283187 22839 leader_election.cc:290] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.283351 22764 raft_consensus.cc:3055] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.283761 22764 raft_consensus.cc:740] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.282773 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.285274 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:46.284543 22838 leader_election.cc:290] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.285569 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840f0061cce4401796a5422bed9ddcef" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.285897 22763 raft_consensus.cc:3055] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.285521 22913 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.286161 22909 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.290503 23030 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:46.292894 22913 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.292711 22764 consensus_queue.cc:260] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.294020 22764 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.294888 22909 raft_consensus.cc:2468] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.295113 22763 raft_consensus.cc:740] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.296281 22802 leader_election.cc:304] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.295991 22763 consensus_queue.cc:260] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.297260 22802 leader_election.cc:304] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.298128 22982 raft_consensus.cc:2804] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.298617 22982 raft_consensus.cc:697] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.298770 22986 raft_consensus.cc:2804] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.304643 22763 raft_consensus.cc:3060] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.305585 22982 consensus_queue.cc:237] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.320634 23030 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:11:46.315413 22764 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.347538 22963 raft_consensus.cc:993] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:46.347236 22763 raft_consensus.cc:2468] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.348064 22963 raft_consensus.cc:1081] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Signalling peer ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:46.350137 22963 raft_consensus.cc:993] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.350549 22963 raft_consensus.cc:1081] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.350633 22909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:41748
I20260709 12:11:46.351212 22909 raft_consensus.cc:493] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.351570 22909 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.348085 22986 raft_consensus.cc:697] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.353920 22986 consensus_queue.cc:237] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.354758 23030 raft_consensus.cc:993] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.355160 23030 raft_consensus.cc:1081] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:46.355847 22763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38428
I20260709 12:11:46.356463 22763 raft_consensus.cc:493] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.356880 22763 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.357121 22839 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38344
I20260709 12:11:46.357946 22839 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.358296 22839 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.364795 22839 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.365135 22658 catalog_manager.cc:5697] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 1 to 2, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 2 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.366923 22658 catalog_manager.cc:5697] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 1 to 2, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 2 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.367817 22839 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.368105 22913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.368655 22913 raft_consensus.cc:3055] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.368963 22913 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:46.369745 22913 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.370922 22913 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.373138 22763 raft_consensus.cc:515] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.375846 22986 raft_consensus.cc:993] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.376363 23041 raft_consensus.cc:1081] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.377841 22763 leader_election.cc:290] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.378855 22910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.379297 22764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:46.379400 22910 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.377579 22909 raft_consensus.cc:515] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.380153 22962 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:46.382102 22909 leader_election.cc:290] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:46.382894 22764 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.385536 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:46.386230 22838 raft_consensus.cc:3055] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.386597 22838 raft_consensus.cc:740] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.387410 22838 consensus_queue.cc:260] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.388258 22762 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38428
I20260709 12:11:46.388790 22762 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.389146 22762 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.389415 22764 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.393443 22913 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:46.394784 22802 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.388767 22838 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.395808 22982 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.396947 22982 raft_consensus.cc:697] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.397765 22982 consensus_queue.cc:237] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.399169 22762 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:46.403360 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.404009 22911 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.409425 22910 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.410552 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ba99eba7854fb59ff6d0c3e4388d94" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:46.411166 22763 raft_consensus.cc:3060] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.412088 22911 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.410869 22727 leader_election.cc:304] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.413201 22962 raft_consensus.cc:2804] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.413676 22962 raft_consensus.cc:697] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.414475 22962 consensus_queue.cc:237] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.418519 22727 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.411612 22839 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:46.419613 22839 raft_consensus.cc:3055] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.419950 22839 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.420840 22839 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.422501 22839 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.423779 22962 raft_consensus.cc:2804] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.424306 22962 raft_consensus.cc:697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.407476 22838 raft_consensus.cc:2468] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 2.
I20260709 12:11:46.425132 22962 consensus_queue.cc:237] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:46.425760 22879 leader_election.cc:304] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:46.426661 22964 raft_consensus.cc:2804] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.427129 22964 raft_consensus.cc:697] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 2 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:46.427925 22964 consensus_queue.cc:237] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.423085 22658 catalog_manager.cc:5697] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 1 to 2, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 2 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.435480 22762 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Requested vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:46.418675 22763 raft_consensus.cc:2468] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 2.
I20260709 12:11:46.441205 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:46.441833 22838 raft_consensus.cc:3055] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.442204 22838 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:46.442142 22962 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:46.450762 22661 catalog_manager.cc:5697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 1 to 2, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 2 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:46.443168 22838 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:46.450765 22839 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.455468 22838 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.471974 22964 raft_consensus.cc:993] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.472440 22964 raft_consensus.cc:1081] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:11:46.462641 22838 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.477746 22763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38440
I20260709 12:11:46.478308 22763 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:46.478686 22763 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.478916 22661 catalog_manager.cc:5697] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 1 to 2, leader changed from ed03529aa1c04c2d9d23061fba504067 (127.22.25.67) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 2 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.486555 22763 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.489508 22763 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:46.489544 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:46.490608 22909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:46.491015 22837 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.491161 22909 raft_consensus.cc:3055] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:46.491493 22909 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:46.493148 22909 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.494426 22909 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:46.501801 22837 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.502583 22909 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 2.
I20260709 12:11:46.503106 22730 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:46.503996 22985 raft_consensus.cc:2804] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:46.504458 22985 raft_consensus.cc:697] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:46.501013 22657 catalog_manager.cc:5697] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 1 to 2, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 2 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:46.505262 22985 consensus_queue.cc:237] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:46.518307 22661 catalog_manager.cc:5697] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 1 to 2, leader changed from ed03529aa1c04c2d9d23061fba504067 (127.22.25.67) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 2 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:46.604301 22909 raft_consensus.cc:1275] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.605839 22982 consensus_queue.cc:1048] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.615890 22763 raft_consensus.cc:1275] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.617858 23041 consensus_queue.cc:1048] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.731518 22763 raft_consensus.cc:1275] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.733716 23041 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.734831 22763 raft_consensus.cc:1275] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.737016 23041 consensus_queue.cc:1048] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.737615 22909 raft_consensus.cc:1275] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.739250 23041 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.757756 22911 raft_consensus.cc:1275] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.762111 22982 consensus_queue.cc:1048] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.810623 22909 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.811985 22985 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.820693 22837 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.822721 22962 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.864935 22763 raft_consensus.cc:1275] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.866065 23030 consensus_queue.cc:1048] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.873293 22837 raft_consensus.cc:1275] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.874750 22964 consensus_queue.cc:1048] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.899132 22909 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.901286 22909 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.901158 23041 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.902433 23024 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.911490 22763 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.912998 23041 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:46.913547 22838 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:46.915189 23024 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:47.084786 22911 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:47.085913 22962 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:47.094511 22838 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:47.096316 22985 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:49.091482 22629 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:11:49.092249 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:49.117714 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:11:49.118237 22837 raft_consensus.cc:606] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Received request to transfer leadership to TS ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:49.130302 22762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:45156
I20260709 12:11:49.131019 22762 raft_consensus.cc:606] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Received request to transfer leadership to TS ed03529aa1c04c2d9d23061fba504067
I20260709 12:11:49.132336 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:11:49.132814 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:49.368902 23059 raft_consensus.cc:993] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: : Instructing follower ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:49.369370 22985 raft_consensus.cc:1081] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Signalling peer ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:49.370688 22909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:41736
I20260709 12:11:49.371181 22909 raft_consensus.cc:493] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:49.371457 22909 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.375896 22909 raft_consensus.cc:515] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.377420 22909 leader_election.cc:290] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 3 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:49.378330 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:49.378523 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:49.378804 22762 raft_consensus.cc:3055] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:49.379033 22838 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.379122 22762 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:49.379900 22762 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.380792 22762 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.384255 22838 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 3.
I20260709 12:11:49.385299 22762 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 3.
I20260709 12:11:49.385396 22879 leader_election.cc:304] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:49.386210 22964 raft_consensus.cc:2804] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:49.387202 22964 raft_consensus.cc:697] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:49.387959 22964 consensus_queue.cc:237] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.396823 22658 catalog_manager.cc:5697] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 2 to 3, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 3 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:49.587762 23067 raft_consensus.cc:993] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:49.588230 23041 raft_consensus.cc:1081] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Signalling peer ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:11:49.589694 22909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:41748
I20260709 12:11:49.590143 22909 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:49.590397 22909 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.594831 22909 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.596909 22909 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 3 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:11:49.597910 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:49.598289 22838 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:11:49.598564 22762 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.598867 22838 raft_consensus.cc:3055] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:49.599184 22838 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:49.600003 22838 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.601250 22838 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:49.605278 22762 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 3.
I20260709 12:11:49.606465 22877 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:49.607254 22964 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:49.607767 22964 raft_consensus.cc:697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:49.607791 22838 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 3.
I20260709 12:11:49.608685 22964 consensus_queue.cc:237] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:49.616014 22658 catalog_manager.cc:5697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 2 to 3, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 3 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:49.967377 22762 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:49.968598 22964 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:49.977118 22838 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:49.978639 23070 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:50.133947 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:50.139238 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:50.139631 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:50.198930 22838 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:50.200516 23070 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:50.208995 22762 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:50.210312 22964 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:51.140547 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:51.145447 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:51.145865 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:52.146790 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:52.151790 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:52.152220 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:53.153326 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:53.158466 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:53.158917 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:54.160034 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:54.164865 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:54.165273 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:55.166283 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:55.171010 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:55.171447 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:56.172333 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:56.177045 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:56.177446 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:57.178361 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:57.183151 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:57.183570 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:58.184530 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:58.189291 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:58.189718 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:59.219511 22629 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:11:59.224399 22762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:59.225020 22762 raft_consensus.cc:606] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:59.230195 22762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:59.230646 22762 raft_consensus.cc:606] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:59.236594 22762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:45140
I20260709 12:11:59.237087 22762 raft_consensus.cc:606] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:59.242306 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:51904
I20260709 12:11:59.242750 22911 raft_consensus.cc:606] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:59.247710 22911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
mode: GRACEFUL
new_leader_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:51904
I20260709 12:11:59.248262 22911 raft_consensus.cc:606] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Received request to transfer leadership to TS 6a94dc37206a4330b414b37ea06b40be
I20260709 12:11:59.254349 23085 raft_consensus.cc:993] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.254865 23085 raft_consensus.cc:1081] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.256501 22837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:59.256971 22837 raft_consensus.cc:493] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:59.257354 22837 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:59.263824 22837 raft_consensus.cc:515] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.265635 22837 leader_election.cc:290] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:59.266641 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:59.266526 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:59.267369 22911 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:59.267414 22762 raft_consensus.cc:3055] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:59.267892 22762 raft_consensus.cc:740] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:59.268684 22762 consensus_queue.cc:260] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.269933 22762 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:59.274363 22911 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:59.275769 22802 leader_election.cc:304] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:59.276772 23084 raft_consensus.cc:2804] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:59.276942 22762 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 2.
I20260709 12:11:59.277956 23084 raft_consensus.cc:697] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:59.278784 23084 consensus_queue.cc:237] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.285975 22657 catalog_manager.cc:5697] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 1 to 2, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 2 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.320111 23079 raft_consensus.cc:993] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.320622 23092 raft_consensus.cc:1081] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.322185 22837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38344
I20260709 12:11:59.322628 22837 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:59.322881 22837 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.327436 22837 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.329207 22837 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 4 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:59.329957 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:59.330224 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:59.330502 22762 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.330701 22911 raft_consensus.cc:3055] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:59.331007 22911 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:59.331737 22911 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.332643 22911 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.335107 22762 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 4.
I20260709 12:11:59.336279 22803 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:59.336975 23084 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:59.337437 22911 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 4.
I20260709 12:11:59.337450 23084 raft_consensus.cc:697] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:59.338510 23084 consensus_queue.cc:237] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.345394 22657 catalog_manager.cc:5697] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 3 to 4, leader changed from ed03529aa1c04c2d9d23061fba504067 (127.22.25.67) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 4 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.484265 23079 raft_consensus.cc:993] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.485361 23092 raft_consensus.cc:1081] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.488066 22837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38344
I20260709 12:11:59.488731 22837 raft_consensus.cc:493] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:59.489143 22837 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.496068 22837 raft_consensus.cc:515] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.499430 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:59.499565 22837 leader_election.cc:290] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 4 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:59.499436 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8ccc590e3e4beaad73bd86c68df22a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:59.500245 22762 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.500398 22911 raft_consensus.cc:3055] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:59.500738 22911 raft_consensus.cc:740] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:11:59.501437 22911 consensus_queue.cc:260] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.502552 22911 raft_consensus.cc:3060] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:59.507944 22762 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 4.
I20260709 12:11:59.508942 22911 raft_consensus.cc:2468] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 4.
I20260709 12:11:59.509187 22803 leader_election.cc:304] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, 6a94dc37206a4330b414b37ea06b40be; no voters: 
I20260709 12:11:59.510316 23084 raft_consensus.cc:2804] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:11:59.510810 23084 raft_consensus.cc:697] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:59.511778 23084 consensus_queue.cc:237] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.520284 22657 catalog_manager.cc:5697] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 3 to 4, leader changed from ed03529aa1c04c2d9d23061fba504067 (127.22.25.67) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 4 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.521059 23077 raft_consensus.cc:993] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.521603 23085 raft_consensus.cc:1081] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.523310 22837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:59.523988 22837 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:59.524399 22837 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.530954 22837 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.533012 22837 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 3 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:11:59.533537 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:59.534113 22762 raft_consensus.cc:3055] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:59.534372 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:59.534530 22762 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:59.534898 22911 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.535822 22762 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.537047 22762 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.540495 22911 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 3.
I20260709 12:11:59.541505 22802 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:59.542130 23084 raft_consensus.cc:2804] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:59.542575 23084 raft_consensus.cc:697] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:59.543567 22762 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 3.
I20260709 12:11:59.543366 23084 consensus_queue.cc:237] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:11:59.550402 22657 catalog_manager.cc:5697] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 2 to 3, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 3 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.570601 23077 raft_consensus.cc:993] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: : Instructing follower 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.571054 23085 raft_consensus.cc:1081] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Signalling peer 6a94dc37206a4330b414b37ea06b40be to start an election
I20260709 12:11:59.572724 22837 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
 from {username='slave'} at 127.0.0.1:38336
I20260709 12:11:59.573407 22837 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:59.573709 22837 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.577956 22837 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:59.579864 22837 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 3 election: Requested vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543)
I20260709 12:11:59.580685 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:11:59.580951 22762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "6a94dc37206a4330b414b37ea06b40be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:11:59.581301 22911 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.581532 22762 raft_consensus.cc:3055] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:59.581826 22762 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:11:59.582458 22762 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:59.583334 22762 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:59.586462 22911 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 3.
I20260709 12:11:59.587433 22802 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a94dc37206a4330b414b37ea06b40be, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:11:59.587482 22762 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a94dc37206a4330b414b37ea06b40be in term 3.
I20260709 12:11:59.588173 23084 raft_consensus.cc:2804] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:59.588551 23084 raft_consensus.cc:697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Becoming Leader. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:11:59.589321 23084 consensus_queue.cc:237] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:11:59.596122 22657 catalog_manager.cc:5697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be reported cstate change: term changed from 2 to 3, leader changed from 57e418e6877547ea92d981055691e9fd (127.22.25.65) to 6a94dc37206a4330b414b37ea06b40be (127.22.25.66). New cstate: current_term: 3 leader_uuid: "6a94dc37206a4330b414b37ea06b40be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:59.691171 22762 raft_consensus.cc:1275] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:59.693734 23084 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:59.704962 22911 raft_consensus.cc:1275] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:59.706449 23100 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:11:59.876538 22762 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:59.877656 23100 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:59.885955 22911 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:59.887466 23086 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:59.935619 22911 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:59.936693 23086 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:59.944993 22762 raft_consensus.cc:1275] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:11:59.946380 23100 consensus_queue.cc:1048] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:59.961001 22763 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:59.962113 23100 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:59.970227 22911 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:59.971520 23086 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:59.982249 22763 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:59.983384 23084 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:59.995128 22911 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Refusing update from remote peer 6a94dc37206a4330b414b37ea06b40be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:59.996495 23084 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:02.287741 22629 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:12:02.288437 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:02.295238 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:12:02.295766 22837 raft_consensus.cc:606] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Received request to transfer leadership to TS ed03529aa1c04c2d9d23061fba504067
I20260709 12:12:02.298237 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:12:02.298735 22837 raft_consensus.cc:606] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:12:02.301052 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:12:02.301528 22837 raft_consensus.cc:606] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:12:02.303826 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:12:02.304265 22837 raft_consensus.cc:606] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Received request to transfer leadership to TS ed03529aa1c04c2d9d23061fba504067
I20260709 12:12:02.306468 22837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
mode: GRACEFUL
new_leader_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:48868
I20260709 12:12:02.306881 22837 raft_consensus.cc:606] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Received request to transfer leadership to TS 57e418e6877547ea92d981055691e9fd
I20260709 12:12:02.307852 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:02.308256 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:02.329988 23112 raft_consensus.cc:993] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.330404 23084 raft_consensus.cc:1081] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.331858 22763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38428
I20260709 12:12:02.332382 22763 raft_consensus.cc:493] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:02.332651 22763 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.336911 22763 raft_consensus.cc:515] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:12:02.338872 22763 leader_election.cc:290] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 4 election: Requested vote from peers ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:12:02.339668 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:12:02.339871 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b2635f595b4b60a95dbbfc8967a5f8" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:12:02.340123 22911 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.340399 22837 raft_consensus.cc:3055] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:02.340649 22837 raft_consensus.cc:740] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:12:02.341213 22837 consensus_queue.cc:260] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:12:02.342015 22837 raft_consensus.cc:3060] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.344307 22911 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 4.
I20260709 12:12:02.345366 22727 leader_election.cc:304] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:12:02.346118 22837 raft_consensus.cc:2468] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 4.
I20260709 12:12:02.346720 23113 raft_consensus.cc:2804] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:02.347681 23113 raft_consensus.cc:697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 4 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:12:02.348488 23113 consensus_queue.cc:237] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } }
I20260709 12:12:02.354970 22658 catalog_manager.cc:5697] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 3 to 4, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 4 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.458488 23084 raft_consensus.cc:993] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.459062 23084 raft_consensus.cc:1081] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.460736 22763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38428
I20260709 12:12:02.461409 22763 raft_consensus.cc:493] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:02.461731 22763 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.466784 22763 raft_consensus.cc:515] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.468606 22763 leader_election.cc:290] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 3 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:12:02.469980 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:12:02.470018 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb3e39dba904fc997edf2087a214803" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:12:02.470665 22911 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.470741 22837 raft_consensus.cc:3055] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:02.471068 22837 raft_consensus.cc:740] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:12:02.471751 22837 consensus_queue.cc:260] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.472707 22837 raft_consensus.cc:3060] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.475524 22911 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 3.
I20260709 12:12:02.476616 22727 leader_election.cc:304] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:12:02.477419 23113 raft_consensus.cc:2804] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:02.477620 22837 raft_consensus.cc:2468] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 3.
I20260709 12:12:02.477771 23113 raft_consensus.cc:697] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 3 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:12:02.478885 23113 consensus_queue.cc:237] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.486199 22658 catalog_manager.cc:5697] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 2 to 3, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 3 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:02.498822 23112 raft_consensus.cc:993] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.499284 23084 raft_consensus.cc:1081] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Signalling peer 57e418e6877547ea92d981055691e9fd to start an election
I20260709 12:12:02.500839 22763 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f"
dest_uuid: "57e418e6877547ea92d981055691e9fd"
 from {username='slave'} at 127.0.0.1:38428
I20260709 12:12:02.501288 22763 raft_consensus.cc:493] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:02.501569 22763 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.506254 22763 raft_consensus.cc:515] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.507718 22763 leader_election.cc:290] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 3 election: Requested vote from peers 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929), ed03529aa1c04c2d9d23061fba504067 (127.22.25.67:43431)
I20260709 12:12:02.508620 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:12:02.508633 22911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cea1a6ad774898ba30a64d0da5987f" candidate_uuid: "57e418e6877547ea92d981055691e9fd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
I20260709 12:12:02.509181 22837 raft_consensus.cc:3055] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:02.509312 22911 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.509472 22837 raft_consensus.cc:740] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:12:02.510196 22837 consensus_queue.cc:260] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.511253 22837 raft_consensus.cc:3060] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:02.514745 23112 raft_consensus.cc:993] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:12:02.514710 22911 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 3.
I20260709 12:12:02.515434 23084 raft_consensus.cc:1081] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Signalling peer ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:12:02.515833 22727 leader_election.cc:304] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:12:02.516551 23113 raft_consensus.cc:2804] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:02.516683 22837 raft_consensus.cc:2468] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e418e6877547ea92d981055691e9fd in term 3.
I20260709 12:12:02.517102 23113 raft_consensus.cc:697] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 3 LEADER]: Becoming Leader. State: Replica: 57e418e6877547ea92d981055691e9fd, State: Running, Role: LEADER
I20260709 12:12:02.517167 22911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:41748
I20260709 12:12:02.517741 22911 raft_consensus.cc:493] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:02.518100 22911 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:02.518055 23113 consensus_queue.cc:237] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.523340 22911 raft_consensus.cc:515] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.524950 22658 catalog_manager.cc:5697] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd reported cstate change: term changed from 2 to 3, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to 57e418e6877547ea92d981055691e9fd (127.22.25.65). New cstate: current_term: 3 leader_uuid: "57e418e6877547ea92d981055691e9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:02.525612 22911 leader_election.cc:290] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 5 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:12:02.526571 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:12:02.526743 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06bfc2e074c54c1a8b464c942281ed08" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:12:02.527195 22763 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:02.527282 22837 raft_consensus.cc:3055] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:02.529950 22837 raft_consensus.cc:740] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:12:02.530814 22837 consensus_queue.cc:260] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.532151 22837 raft_consensus.cc:3060] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:02.537701 22763 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 5.
I20260709 12:12:02.537734 22837 raft_consensus.cc:2468] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 5.
I20260709 12:12:02.539053 22877 leader_election.cc:304] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:12:02.539769 23079 raft_consensus.cc:2804] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:02.540707 23079 raft_consensus.cc:697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 5 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:12:02.541431 23079 consensus_queue.cc:237] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.548595 22658 catalog_manager.cc:5697] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 4 to 5, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 5 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.572715 23112 raft_consensus.cc:993] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: : Instructing follower ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:12:02.573221 23084 raft_consensus.cc:1081] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Signalling peer ed03529aa1c04c2d9d23061fba504067 to start an election
I20260709 12:12:02.574824 22911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a"
dest_uuid: "ed03529aa1c04c2d9d23061fba504067"
 from {username='slave'} at 127.0.0.1:41748
I20260709 12:12:02.575299 22911 raft_consensus.cc:493] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:02.575676 22911 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.580035 22911 raft_consensus.cc:515] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.581449 22911 leader_election.cc:290] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 4 election: Requested vote from peers 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543), 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929)
I20260709 12:12:02.582257 22763 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "57e418e6877547ea92d981055691e9fd"
I20260709 12:12:02.582347 22837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a96aa0ed6654a62a92c20a4b68a530a" candidate_uuid: "ed03529aa1c04c2d9d23061fba504067" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6a94dc37206a4330b414b37ea06b40be"
I20260709 12:12:02.582968 22763 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.582969 22837 raft_consensus.cc:3055] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:02.583421 22837 raft_consensus.cc:740] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a94dc37206a4330b414b37ea06b40be, State: Running, Role: LEADER
I20260709 12:12:02.584084 22837 consensus_queue.cc:260] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.584942 22837 raft_consensus.cc:3060] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:02.587219 22763 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 4.
I20260709 12:12:02.588269 22877 leader_election.cc:304] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e418e6877547ea92d981055691e9fd, ed03529aa1c04c2d9d23061fba504067; no voters: 
I20260709 12:12:02.588863 23079 raft_consensus.cc:2804] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:02.589097 22837 raft_consensus.cc:2468] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed03529aa1c04c2d9d23061fba504067 in term 4.
I20260709 12:12:02.589233 23079 raft_consensus.cc:697] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 4 LEADER]: Becoming Leader. State: Replica: ed03529aa1c04c2d9d23061fba504067, State: Running, Role: LEADER
I20260709 12:12:02.590363 23079 consensus_queue.cc:237] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } }
I20260709 12:12:02.597405 22658 catalog_manager.cc:5697] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 reported cstate change: term changed from 3 to 4, leader changed from 6a94dc37206a4330b414b37ea06b40be (127.22.25.66) to ed03529aa1c04c2d9d23061fba504067 (127.22.25.67). New cstate: current_term: 4 leader_uuid: "ed03529aa1c04c2d9d23061fba504067" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.837877 22837 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:02.839051 23113 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:02.850822 22911 raft_consensus.cc:1275] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:02.852304 23114 consensus_queue.cc:1048] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:02.988751 22763 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:02.989883 23079 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:02.998701 22837 raft_consensus.cc:1275] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:03.000813 23132 consensus_queue.cc:1048] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:03.015867 22763 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 5 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:03.016887 23132 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e418e6877547ea92d981055691e9fd" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 45543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:03.025205 22837 raft_consensus.cc:1275] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 5 FOLLOWER]: Refusing update from remote peer ed03529aa1c04c2d9d23061fba504067: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:03.026629 23132 consensus_queue.cc:1048] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:03.034951 22911 raft_consensus.cc:1275] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:03.036058 23114 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:03.045455 22837 raft_consensus.cc:1275] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:03.047080 23113 consensus_queue.cc:1048] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:03.086915 22911 raft_consensus.cc:1275] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:03.088006 23138 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed03529aa1c04c2d9d23061fba504067" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:03.097558 22837 raft_consensus.cc:1275] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Refusing update from remote peer 57e418e6877547ea92d981055691e9fd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:03.098948 23138 consensus_queue.cc:1048] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a94dc37206a4330b414b37ea06b40be" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:03.309600 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:03.314482 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:03.314859 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:04.315950 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:04.321117 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:04.321583 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:05.322755 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:05.327661 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:05.328060 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:06.329053 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:06.333992 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:06.334630 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:07.336175 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:07.342418 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:07.342810 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:08.343906 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:08.348692 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:08.349093 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:09.350063 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:09.354959 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:09.355429 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:10.356484 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:10.361194 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:10.361620 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:11.362655 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:11.367431 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:11.367882 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:12.421798 22629 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:12:12.422690 22629 tablet_server.cc:179] TabletServer@127.22.25.65:0 shutting down...
I20260709 12:12:12.454218 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:12.455188 22629 tablet_replica.cc:333] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.456606 22629 raft_consensus.cc:2243] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.457566 22629 raft_consensus.cc:2272] T f02ff32889134e3c97a80a38a5d70b01 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.461123 22629 tablet_replica.cc:333] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.461854 22629 raft_consensus.cc:2243] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.462410 22629 raft_consensus.cc:2272] T 6a96aa0ed6654a62a92c20a4b68a530a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.464321 22629 tablet_replica.cc:333] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.464861 22629 raft_consensus.cc:2243] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.465366 22629 raft_consensus.cc:2272] T 54ba99eba7854fb59ff6d0c3e4388d94 P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.467154 22629 tablet_replica.cc:333] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.467873 22629 raft_consensus.cc:2243] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.469137 22629 raft_consensus.cc:2272] T 27cea1a6ad774898ba30a64d0da5987f P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.471282 22629 tablet_replica.cc:333] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.471988 22629 raft_consensus.cc:2243] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.472662 22629 raft_consensus.cc:2272] T 840f0061cce4401796a5422bed9ddcef P 57e418e6877547ea92d981055691e9fd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.474977 22629 tablet_replica.cc:333] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.475569 22629 raft_consensus.cc:2243] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.476713 22629 raft_consensus.cc:2272] T 2cb3e39dba904fc997edf2087a214803 P 57e418e6877547ea92d981055691e9fd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.478816 22629 tablet_replica.cc:333] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.479527 22629 raft_consensus.cc:2243] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.480545 22629 raft_consensus.cc:2272] T 2f8ccc590e3e4beaad73bd86c68df22a P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.482494 22629 tablet_replica.cc:333] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.482999 22629 raft_consensus.cc:2243] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.483678 22629 raft_consensus.cc:2272] T 06bfc2e074c54c1a8b464c942281ed08 P 57e418e6877547ea92d981055691e9fd [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.485813 22629 tablet_replica.cc:333] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd: stopping tablet replica
I20260709 12:12:12.486424 22629 raft_consensus.cc:2243] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.488044 22629 raft_consensus.cc:2272] T d5b2635f595b4b60a95dbbfc8967a5f8 P 57e418e6877547ea92d981055691e9fd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.570011 22629 tablet_server.cc:196] TabletServer@127.22.25.65:0 shutdown complete.
I20260709 12:12:12.592623 22629 tablet_server.cc:179] TabletServer@127.22.25.66:0 shutting down...
I20260709 12:12:12.627895 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:12.628841 22629 tablet_replica.cc:333] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.629663 22629 raft_consensus.cc:2243] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.630785 22629 raft_consensus.cc:2272] T f02ff32889134e3c97a80a38a5d70b01 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.632678 22629 tablet_replica.cc:333] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.633387 22629 raft_consensus.cc:2243] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.634045 22629 raft_consensus.cc:2272] T 54ba99eba7854fb59ff6d0c3e4388d94 P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.636186 22629 tablet_replica.cc:333] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.636898 22629 raft_consensus.cc:2243] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 4 LEADER]: Raft consensus shutting down.
W20260709 12:12:12.637843 22879 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:12:12.638368 22629 raft_consensus.cc:2272] T 2f8ccc590e3e4beaad73bd86c68df22a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.640604 22629 tablet_replica.cc:333] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.641327 22629 raft_consensus.cc:2243] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:12:12.641608 22879 consensus_peers.cc:597] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 -> Peer 6a94dc37206a4330b414b37ea06b40be (127.22.25.66:44929): Couldn't send request to peer 6a94dc37206a4330b414b37ea06b40be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:12.641919 22629 raft_consensus.cc:2272] T 2cb3e39dba904fc997edf2087a214803 P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.643719 22629 tablet_replica.cc:333] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.644196 22629 raft_consensus.cc:2243] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.645006 22629 raft_consensus.cc:2272] T 840f0061cce4401796a5422bed9ddcef P 6a94dc37206a4330b414b37ea06b40be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.646700 22629 tablet_replica.cc:333] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.647183 22629 raft_consensus.cc:2243] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.647703 22629 raft_consensus.cc:2272] T 27cea1a6ad774898ba30a64d0da5987f P 6a94dc37206a4330b414b37ea06b40be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.649468 22629 tablet_replica.cc:333] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.649935 22629 raft_consensus.cc:2243] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.650348 22629 raft_consensus.cc:2272] T 06bfc2e074c54c1a8b464c942281ed08 P 6a94dc37206a4330b414b37ea06b40be [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.652040 22629 tablet_replica.cc:333] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.652482 22629 raft_consensus.cc:2243] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.652961 22629 raft_consensus.cc:2272] T d5b2635f595b4b60a95dbbfc8967a5f8 P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.656018 22629 tablet_replica.cc:333] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be: stopping tablet replica
I20260709 12:12:12.656776 22629 raft_consensus.cc:2243] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.657517 22629 raft_consensus.cc:2272] T 6a96aa0ed6654a62a92c20a4b68a530a P 6a94dc37206a4330b414b37ea06b40be [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:12.661914 22877 consensus_peers.cc:597] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 -> Peer 57e418e6877547ea92d981055691e9fd (127.22.25.65:45543): Couldn't send request to peer 57e418e6877547ea92d981055691e9fd. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:45543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:12.687100 22629 tablet_server.cc:196] TabletServer@127.22.25.66:0 shutdown complete.
I20260709 12:12:12.706593 22629 tablet_server.cc:179] TabletServer@127.22.25.67:0 shutting down...
I20260709 12:12:12.738183 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:12.739117 22629 tablet_replica.cc:333] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.739938 22629 raft_consensus.cc:2243] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.740461 22629 raft_consensus.cc:2272] T f02ff32889134e3c97a80a38a5d70b01 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.742208 22629 tablet_replica.cc:333] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.742682 22629 raft_consensus.cc:2243] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.743479 22629 raft_consensus.cc:2272] T 54ba99eba7854fb59ff6d0c3e4388d94 P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.745244 22629 tablet_replica.cc:333] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.745730 22629 raft_consensus.cc:2243] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.746218 22629 raft_consensus.cc:2272] T 27cea1a6ad774898ba30a64d0da5987f P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.747972 22629 tablet_replica.cc:333] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.748481 22629 raft_consensus.cc:2243] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.748965 22629 raft_consensus.cc:2272] T 2cb3e39dba904fc997edf2087a214803 P ed03529aa1c04c2d9d23061fba504067 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.750661 22629 tablet_replica.cc:333] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.751122 22629 raft_consensus.cc:2243] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.751480 22629 raft_consensus.cc:2272] T 840f0061cce4401796a5422bed9ddcef P ed03529aa1c04c2d9d23061fba504067 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.753108 22629 tablet_replica.cc:333] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.753553 22629 raft_consensus.cc:2243] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.754354 22629 raft_consensus.cc:2272] T 6a96aa0ed6654a62a92c20a4b68a530a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.756392 22629 tablet_replica.cc:333] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.757004 22629 raft_consensus.cc:2243] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:12.758178 22629 raft_consensus.cc:2272] T 06bfc2e074c54c1a8b464c942281ed08 P ed03529aa1c04c2d9d23061fba504067 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.759855 22629 tablet_replica.cc:333] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.760309 22629 raft_consensus.cc:2243] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.760735 22629 raft_consensus.cc:2272] T d5b2635f595b4b60a95dbbfc8967a5f8 P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.762778 22629 tablet_replica.cc:333] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067: stopping tablet replica
I20260709 12:12:12.763330 22629 raft_consensus.cc:2243] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:12.763999 22629 raft_consensus.cc:2272] T 2f8ccc590e3e4beaad73bd86c68df22a P ed03529aa1c04c2d9d23061fba504067 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:12.791505 22629 tablet_server.cc:196] TabletServer@127.22.25.67:0 shutdown complete.
I20260709 12:12:12.810202 22629 master.cc:562] Master@127.22.25.126:45525 shutting down...
I20260709 12:12:13.014796 22629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:12:13.015465 22629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:13.015802 22629 tablet_replica.cc:333] T 00000000000000000000000000000000 P 23d5b23a357149e7991ed3c7f5ce159b: stopping tablet replica
I20260709 12:12:13.036289 22629 master.cc:584] Master@127.22.25.126:45525 shutdown complete.
57e418e6877547ea92d981055691e9fd  4
6a94dc37206a4330b414b37ea06b40be  4
ed03529aa1c04c2d9d23061fba504067  1
57e418e6877547ea92d981055691e9fd  3
6a94dc37206a4330b414b37ea06b40be  3
ed03529aa1c04c2d9d23061fba504067  3
57e418e6877547ea92d981055691e9fd  0
6a94dc37206a4330b414b37ea06b40be  8
ed03529aa1c04c2d9d23061fba504067  1
57e418e6877547ea92d981055691e9fd  3
6a94dc37206a4330b414b37ea06b40be  3
ed03529aa1c04c2d9d23061fba504067  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (30766 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:12:13.080250 22629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.25.126:44009
I20260709 12:12:13.081426 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:13.087148 23154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.087152 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.089205 23157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:13.089520 22629 server_base.cc:1061] running on GCE node
I20260709 12:12:13.090536 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:13.090711 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:13.090875 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599133090856 us; error 0 us; skew 500 ppm
I20260709 12:12:13.091406 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:13.094082 22629 webserver.cc:533] Webserver started at http://127.22.25.126:38939/ using document root <none> and password file <none>
I20260709 12:12:13.094564 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:13.094722 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:13.095011 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:13.096164 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/master-0-root/instance:
uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.100620 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:12:13.104058 23162 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.104807 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:12:13.105088 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/master-0-root
uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.105358 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:13.127434 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:13.128707 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:13.170452 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.126:44009
I20260709 12:12:13.170555 23213 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.126:44009 every 8 connection(s)
I20260709 12:12:13.175177 23214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.186414 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9: Bootstrap starting.
I20260709 12:12:13.191110 23214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:13.195677 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9: No bootstrap required, opened a new log
I20260709 12:12:13.197820 23214 raft_consensus.cc:359] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER }
I20260709 12:12:13.198266 23214 raft_consensus.cc:385] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:13.198444 23214 raft_consensus.cc:740] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd11f7ba0bc48908eeb3acbdea7d9b9, State: Initialized, Role: FOLLOWER
I20260709 12:12:13.198977 23214 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER }
I20260709 12:12:13.199851 23214 raft_consensus.cc:399] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:12:13.200109 23214 raft_consensus.cc:493] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:12:13.200333 23214 raft_consensus.cc:3060] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:13.204900 23214 raft_consensus.cc:515] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER }
I20260709 12:12:13.205444 23214 leader_election.cc:304] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd11f7ba0bc48908eeb3acbdea7d9b9; no voters: 
I20260709 12:12:13.206547 23214 leader_election.cc:290] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:12:13.206911 23217 raft_consensus.cc:2804] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:13.208266 23217 raft_consensus.cc:697] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 1 LEADER]: Becoming Leader. State: Replica: abd11f7ba0bc48908eeb3acbdea7d9b9, State: Running, Role: LEADER
I20260709 12:12:13.208875 23217 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER }
I20260709 12:12:13.209403 23214 sys_catalog.cc:565] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:12:13.213146 23218 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER } }
I20260709 12:12:13.214608 23218 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:13.214080 23219 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd11f7ba0bc48908eeb3acbdea7d9b9. Latest consensus state: current_term: 1 leader_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd11f7ba0bc48908eeb3acbdea7d9b9" member_type: VOTER } }
I20260709 12:12:13.215687 23219 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:13.216688 23224 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:12:13.222913 23224 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:12:13.225894 22629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:12:13.232267 23224 catalog_manager.cc:1383] Generated new cluster ID: 48ec922518cc4c4c9356f8d63adf3172
I20260709 12:12:13.232615 23224 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:12:13.254803 23224 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:12:13.256053 23224 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:12:13.276229 23224 catalog_manager.cc:6070] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9: Generated new TSK 0
I20260709 12:12:13.276860 23224 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:12:13.293349 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:13.299222 23235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.299747 23236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.301918 23238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:13.302079 22629 server_base.cc:1061] running on GCE node
I20260709 12:12:13.303084 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:13.303565 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:13.303802 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599133303783 us; error 0 us; skew 500 ppm
I20260709 12:12:13.304545 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:13.307281 22629 webserver.cc:533] Webserver started at http://127.22.25.65:40639/ using document root <none> and password file <none>
I20260709 12:12:13.307842 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:13.308024 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:13.308287 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:13.309376 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-0-root/instance:
uuid: "95f0c264b4dc4fe28d874331ff366741"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.314069 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:12:13.317667 23243 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.318430 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:12:13.318723 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
uuid: "95f0c264b4dc4fe28d874331ff366741"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.318995 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:13.337093 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:13.338356 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:13.339946 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:13.342626 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:13.342849 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.343327 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:13.343546 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.387126 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.65:36559
I20260709 12:12:13.387241 23305 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.65:36559 every 8 connection(s)
I20260709 12:12:13.392246 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:13.400043 23310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.401067 23311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.404527 23313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:13.405648 22629 server_base.cc:1061] running on GCE node
I20260709 12:12:13.407116 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:13.407382 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:13.407544 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599133407528 us; error 0 us; skew 500 ppm
I20260709 12:12:13.408197 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:13.408206 23306 heartbeater.cc:344] Connected to a master server at 127.22.25.126:44009
I20260709 12:12:13.408649 23306 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:13.409574 23306 heartbeater.cc:507] Master 127.22.25.126:44009 requested a full tablet report, sending...
I20260709 12:12:13.410979 22629 webserver.cc:533] Webserver started at http://127.22.25.66:33907/ using document root <none> and password file <none>
I20260709 12:12:13.411556 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:13.411818 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:13.411876 23179 ts_manager.cc:194] Registered new tserver with Master: 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559)
I20260709 12:12:13.412118 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:13.413354 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-1-root/instance:
uuid: "b4d5867a2db442d291806a1ac0039588"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.413722 23179 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40730
I20260709 12:12:13.418143 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:12:13.421820 23318 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.422753 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:12:13.423059 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
uuid: "b4d5867a2db442d291806a1ac0039588"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.423339 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:13.433320 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:13.434578 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:13.436054 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:13.438279 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:13.438501 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.438720 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:13.438869 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.482921 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.66:37367
I20260709 12:12:13.483047 23380 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.66:37367 every 8 connection(s)
I20260709 12:12:13.488757 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:13.496244 23384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:13.497021 23385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:13.499961 22629 server_base.cc:1061] running on GCE node
W20260709 12:12:13.500272 23387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:13.501381 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:13.501653 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:13.501865 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599133501836 us; error 0 us; skew 500 ppm
I20260709 12:12:13.502405 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:13.502857 23381 heartbeater.cc:344] Connected to a master server at 127.22.25.126:44009
I20260709 12:12:13.503227 23381 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:13.504137 23381 heartbeater.cc:507] Master 127.22.25.126:44009 requested a full tablet report, sending...
I20260709 12:12:13.504990 22629 webserver.cc:533] Webserver started at http://127.22.25.67:40203/ using document root <none> and password file <none>
I20260709 12:12:13.505587 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:13.505820 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:13.506078 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:13.506309 23179 ts_manager.cc:194] Registered new tserver with Master: b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:13.507414 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-2-root/instance:
uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.507907 23179 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40742
I20260709 12:12:13.512620 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:12:13.516500 23392 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.517511 22629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:12:13.517926 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
format_stamp: "Formatted at 2026-07-09 12:12:13 on dist-test-slave-sc22"
I20260709 12:12:13.518260 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599102143852-22629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:13.545599 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:13.546823 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:13.548431 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:13.550711 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:13.550920 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.551151 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:13.551304 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:13.595924 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.67:41953
I20260709 12:12:13.596052 23454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.67:41953 every 8 connection(s)
I20260709 12:12:13.609802 23455 heartbeater.cc:344] Connected to a master server at 127.22.25.126:44009
I20260709 12:12:13.610169 23455 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:13.611022 23455 heartbeater.cc:507] Master 127.22.25.126:44009 requested a full tablet report, sending...
I20260709 12:12:13.613031 23179 ts_manager.cc:194] Registered new tserver with Master: bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:13.613109 22629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013966635s
I20260709 12:12:13.614771 23179 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40744
I20260709 12:12:13.640045 23179 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40750:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:12:13.642956 23179 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:12:13.713689 23271 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae90aa7d14a48a4ace7673c6f365c06 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:13.715229 23271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae90aa7d14a48a4ace7673c6f365c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.718427 23270 tablet_service.cc:1511] Processing CreateTablet for tablet 03866111a0a54ab2ad035e9c53d7c6ce (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:13.719874 23270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03866111a0a54ab2ad035e9c53d7c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.724365 23420 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae90aa7d14a48a4ace7673c6f365c06 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:13.725733 23420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae90aa7d14a48a4ace7673c6f365c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.730989 23419 tablet_service.cc:1511] Processing CreateTablet for tablet 03866111a0a54ab2ad035e9c53d7c6ce (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:13.732389 23419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03866111a0a54ab2ad035e9c53d7c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.736791 23418 tablet_service.cc:1511] Processing CreateTablet for tablet acc1b8b7fb534a65a834a1ad47b92d82 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:13.738292 23418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc1b8b7fb534a65a834a1ad47b92d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.741793 23417 tablet_service.cc:1511] Processing CreateTablet for tablet 48cde435c4ef4b40b4c2c179b305b37e (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:13.743249 23417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cde435c4ef4b40b4c2c179b305b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.747459 23416 tablet_service.cc:1511] Processing CreateTablet for tablet 4908b7f19700429584ac45090f98580d (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:13.749287 23416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4908b7f19700429584ac45090f98580d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.750764 23470 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:13.753021 23415 tablet_service.cc:1511] Processing CreateTablet for tablet a697b070e4cf4d599a7959216dafa9b7 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:13.754464 23415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a697b070e4cf4d599a7959216dafa9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.758311 23414 tablet_service.cc:1511] Processing CreateTablet for tablet a0ead34498294751a5df537c369db528 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:13.759868 23414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ead34498294751a5df537c369db528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.763674 23413 tablet_service.cc:1511] Processing CreateTablet for tablet a60afbcca2b847fe93f7451e37f98b79 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:13.764552 23271 tablet_service.cc:1511] Processing CreateTablet for tablet 48cde435c4ef4b40b4c2c179b305b37e (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:13.765036 23413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60afbcca2b847fe93f7451e37f98b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.765807 23271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cde435c4ef4b40b4c2c179b305b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.766609 23268 tablet_service.cc:1511] Processing CreateTablet for tablet a697b070e4cf4d599a7959216dafa9b7 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:13.767911 23268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a697b070e4cf4d599a7959216dafa9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.758391 23470 tablet_bootstrap.cc:654] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:13.771095 23267 tablet_service.cc:1511] Processing CreateTablet for tablet a0ead34498294751a5df537c369db528 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:13.772619 23267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ead34498294751a5df537c369db528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.776891 23266 tablet_service.cc:1511] Processing CreateTablet for tablet a60afbcca2b847fe93f7451e37f98b79 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:13.779951 23270 tablet_service.cc:1511] Processing CreateTablet for tablet acc1b8b7fb534a65a834a1ad47b92d82 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:13.783303 23269 tablet_service.cc:1511] Processing CreateTablet for tablet 4908b7f19700429584ac45090f98580d (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:13.778600 23266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60afbcca2b847fe93f7451e37f98b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.792011 23269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4908b7f19700429584ac45090f98580d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.808207 23270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc1b8b7fb534a65a834a1ad47b92d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.811050 23265 tablet_service.cc:1511] Processing CreateTablet for tablet bf956843dd6547a19fd10e4fd69dc0bb (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:13.812412 23265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf956843dd6547a19fd10e4fd69dc0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.812796 23346 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae90aa7d14a48a4ace7673c6f365c06 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:13.814147 23346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae90aa7d14a48a4ace7673c6f365c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.815112 23419 tablet_service.cc:1511] Processing CreateTablet for tablet bf956843dd6547a19fd10e4fd69dc0bb (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:13.816447 23419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf956843dd6547a19fd10e4fd69dc0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.816778 23264 tablet_service.cc:1511] Processing CreateTablet for tablet c6a841b5028c46c0927d865e58b0f3c9 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:13.818104 23264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a841b5028c46c0927d865e58b0f3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.816799 23345 tablet_service.cc:1511] Processing CreateTablet for tablet 03866111a0a54ab2ad035e9c53d7c6ce (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:13.821344 23345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03866111a0a54ab2ad035e9c53d7c6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.819327 23343 tablet_service.cc:1511] Processing CreateTablet for tablet 48cde435c4ef4b40b4c2c179b305b37e (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:13.825412 23343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cde435c4ef4b40b4c2c179b305b37e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.830691 23342 tablet_service.cc:1511] Processing CreateTablet for tablet 4908b7f19700429584ac45090f98580d (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:13.832922 23341 tablet_service.cc:1511] Processing CreateTablet for tablet a697b070e4cf4d599a7959216dafa9b7 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:13.834676 23341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a697b070e4cf4d599a7959216dafa9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.836632 23471 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:13.837014 23340 tablet_service.cc:1511] Processing CreateTablet for tablet a0ead34498294751a5df537c369db528 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:13.838670 23340 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0ead34498294751a5df537c369db528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.841145 23339 tablet_service.cc:1511] Processing CreateTablet for tablet a60afbcca2b847fe93f7451e37f98b79 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:13.818053 23344 tablet_service.cc:1511] Processing CreateTablet for tablet acc1b8b7fb534a65a834a1ad47b92d82 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:13.862241 23471 tablet_bootstrap.cc:654] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:13.868988 23339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60afbcca2b847fe93f7451e37f98b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.873838 23414 tablet_service.cc:1511] Processing CreateTablet for tablet c6a841b5028c46c0927d865e58b0f3c9 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:13.874074 23337 tablet_service.cc:1511] Processing CreateTablet for tablet c6a841b5028c46c0927d865e58b0f3c9 (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:13.875291 23414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a841b5028c46c0927d865e58b0f3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.876716 23338 tablet_service.cc:1511] Processing CreateTablet for tablet bf956843dd6547a19fd10e4fd69dc0bb (DEFAULT_TABLE table=test-workload [id=3902934c4a924e7984b5d3c9b556f36a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:13.879338 23344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acc1b8b7fb534a65a834a1ad47b92d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.892020 23342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4908b7f19700429584ac45090f98580d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.906315 23470 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:13.906900 23470 ts_tablet_manager.cc:1403] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.156s	user 0.025s	sys 0.009s
I20260709 12:12:13.910027 23470 raft_consensus.cc:359] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:13.911331 23470 raft_consensus.cc:385] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:13.911780 23470 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:13.912844 23470 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:13.915077 23337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a841b5028c46c0927d865e58b0f3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.916298 23338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf956843dd6547a19fd10e4fd69dc0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:13.919507 23474 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:13.921468 23471 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:13.930661 23474 tablet_bootstrap.cc:654] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:13.931231 23471 ts_tablet_manager.cc:1403] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.017s
I20260709 12:12:13.934687 23471 raft_consensus.cc:359] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:13.942387 23306 heartbeater.cc:499] Master 127.22.25.126:44009 was elected leader, sending a full tablet report...
I20260709 12:12:13.935552 23471 raft_consensus.cc:385] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:13.945834 23470 ts_tablet_manager.cc:1434] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.008s
I20260709 12:12:13.950754 23475 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.958449 23471 raft_consensus.cc:740] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:13.960866 23475 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:13.965282 23474 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:13.966058 23475 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:13.972209 23470 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:13.965929 23474 ts_tablet_manager.cc:1403] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.003s
I20260709 12:12:13.973192 23471 consensus_queue.cc:260] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:13.988619 23470 tablet_bootstrap.cc:654] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:13.990507 23455 heartbeater.cc:499] Master 127.22.25.126:44009 was elected leader, sending a full tablet report...
I20260709 12:12:13.990373 23474 raft_consensus.cc:359] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:13.991180 23474 raft_consensus.cc:385] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:13.998869 23471 ts_tablet_manager.cc:1434] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 12:12:13.991497 23474 raft_consensus.cc:740] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.002068 23474 consensus_queue.cc:260] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.006325 23381 heartbeater.cc:499] Master 127.22.25.126:44009 was elected leader, sending a full tablet report...
I20260709 12:12:14.000186 23471 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.012933 23480 raft_consensus.cc:493] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.013540 23480 raft_consensus.cc:515] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.016817 23474 ts_tablet_manager.cc:1434] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:12:14.017668 23474 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.018585 23470 tablet_bootstrap.cc:492] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.019093 23470 ts_tablet_manager.cc:1403] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.016s
I20260709 12:12:14.018993 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
W20260709 12:12:14.020962 23245 leader_election.cc:343] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.021247 23471 tablet_bootstrap.cc:654] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.022220 23470 raft_consensus.cc:359] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.022945 23470 raft_consensus.cc:385] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.023270 23474 tablet_bootstrap.cc:654] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.023267 23470 raft_consensus.cc:740] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.024402 23470 consensus_queue.cc:260] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.028375 23470 ts_tablet_manager.cc:1434] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:12:14.029381 23470 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.035475 23470 tablet_bootstrap.cc:654] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.045393 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.046619 23281 raft_consensus.cc:2468] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 0.
I20260709 12:12:14.048673 23394 leader_election.cc:304] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.048633 23470 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.050460 23470 ts_tablet_manager.cc:1403] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 12:12:14.054442 23480 leader_election.cc:290] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.055234 23480 raft_consensus.cc:2804] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.057349 23470 raft_consensus.cc:359] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.061601 23470 raft_consensus.cc:385] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.062019 23470 raft_consensus.cc:740] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.062803 23470 consensus_queue.cc:260] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.061528 23480 raft_consensus.cc:493] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.063861 23480 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.070963 23480 raft_consensus.cc:515] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.073266 23470 ts_tablet_manager.cc:1434] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:12:14.076004 23470 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.082100 23470 tablet_bootstrap.cc:654] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.087098 23480 leader_election.cc:290] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.087064 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.087815 23281 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.101912 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:14.102820 23355 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.104056 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.114321 23355 raft_consensus.cc:2468] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 1.
I20260709 12:12:14.115962 23394 leader_election.cc:304] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.117136 23480 raft_consensus.cc:2804] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.117612 23480 raft_consensus.cc:697] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:14.118422 23480 consensus_queue.cc:237] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:14.122865 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
W20260709 12:12:14.125017 23246 leader_election.cc:343] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.125869 23246 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741; no voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.126791 23475 raft_consensus.cc:2749] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.128036 23281 raft_consensus.cc:2468] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 1.
I20260709 12:12:14.134778 23470 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.135290 23470 ts_tablet_manager.cc:1403] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.059s	user 0.029s	sys 0.015s
I20260709 12:12:14.135711 23474 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.136278 23474 ts_tablet_manager.cc:1403] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.119s	user 0.034s	sys 0.077s
I20260709 12:12:14.135557 23176 catalog_manager.cc:5697] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 1 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.138339 23470 raft_consensus.cc:359] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.139143 23470 raft_consensus.cc:385] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.139613 23470 raft_consensus.cc:740] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.139161 23474 raft_consensus.cc:359] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.140141 23474 raft_consensus.cc:385] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.140463 23474 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.140440 23470 consensus_queue.cc:260] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.141815 23474 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.143110 23471 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.143676 23471 ts_tablet_manager.cc:1403] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.144s	user 0.038s	sys 0.066s
I20260709 12:12:14.143959 23474 ts_tablet_manager.cc:1434] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:14.144490 23470 ts_tablet_manager.cc:1434] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:12:14.144896 23474 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.145460 23470 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.146821 23482 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.146919 23471 raft_consensus.cc:359] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.147344 23482 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.147794 23471 raft_consensus.cc:385] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.148171 23471 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.149235 23471 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.151578 23471 ts_tablet_manager.cc:1434] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:12:14.152221 23474 tablet_bootstrap.cc:654] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.152688 23471 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.152657 23482 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.159852 23471 tablet_bootstrap.cc:654] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.175572 23470 tablet_bootstrap.cc:654] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.187121 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
W20260709 12:12:14.189865 23320 leader_election.cc:343] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.196550 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.198149 23474 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
W20260709 12:12:14.198223 23321 leader_election.cc:343] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.198746 23474 ts_tablet_manager.cc:1403] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.024s
I20260709 12:12:14.198791 23471 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.198742 23321 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4d5867a2db442d291806a1ac0039588; no voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.199194 23470 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.199678 23471 ts_tablet_manager.cc:1403] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.021s
I20260709 12:12:14.199731 23470 ts_tablet_manager.cc:1403] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 12:12:14.199806 23482 raft_consensus.cc:2749] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.201645 23474 raft_consensus.cc:359] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.202179 23474 raft_consensus.cc:385] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.202435 23474 raft_consensus.cc:740] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.202644 23471 raft_consensus.cc:359] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.202864 23470 raft_consensus.cc:359] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.203416 23471 raft_consensus.cc:385] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.203083 23474 consensus_queue.cc:260] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.203545 23470 raft_consensus.cc:385] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.203768 23471 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.203879 23470 raft_consensus.cc:740] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.204774 23470 consensus_queue.cc:260] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.205528 23474 ts_tablet_manager.cc:1434] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:14.207489 23470 ts_tablet_manager.cc:1434] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:14.208565 23474 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.208739 23470 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.205641 23471 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.217334 23474 tablet_bootstrap.cc:654] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.217831 23470 tablet_bootstrap.cc:654] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.219468 23471 ts_tablet_manager.cc:1434] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:12:14.221179 23471 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
W20260709 12:12:14.226256 23227 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:14.227803 23471 tablet_bootstrap.cc:654] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.239468 23470 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.239938 23470 ts_tablet_manager.cc:1403] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.012s
I20260709 12:12:14.240320 23474 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.240804 23474 ts_tablet_manager.cc:1403] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.017s
I20260709 12:12:14.242275 23471 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.242087 23470 raft_consensus.cc:359] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.242664 23470 raft_consensus.cc:385] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.242796 23471 ts_tablet_manager.cc:1403] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 12:12:14.242935 23470 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.243709 23470 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.243741 23474 raft_consensus.cc:359] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.244607 23474 raft_consensus.cc:385] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.244933 23474 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.245965 23470 ts_tablet_manager.cc:1434] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 12:12:14.245697 23474 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.245894 23471 raft_consensus.cc:359] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.246608 23471 raft_consensus.cc:385] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.247032 23471 raft_consensus.cc:740] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.247359 23470 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.248461 23474 ts_tablet_manager.cc:1434] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:14.248169 23471 consensus_queue.cc:260] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.249450 23474 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.252832 23471 ts_tablet_manager.cc:1434] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:12:14.253927 23471 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.254266 23470 tablet_bootstrap.cc:654] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.255559 23475 raft_consensus.cc:493] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.256532 23474 tablet_bootstrap.cc:654] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.256325 23475 raft_consensus.cc:515] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.258579 23475 leader_election.cc:290] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.260509 23471 tablet_bootstrap.cc:654] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.261149 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.263630 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.264389 23430 raft_consensus.cc:2468] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.266842 23480 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.267335 23480 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.267803 23246 leader_election.cc:304] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
W20260709 12:12:14.268787 23245 leader_election.cc:343] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.269389 23480 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.271219 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.271864 23281 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 0.
I20260709 12:12:14.273051 23394 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.274537 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
W20260709 12:12:14.275905 23394 leader_election.cc:343] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.276742 23480 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.277127 23480 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.277436 23480 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.284324 23480 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.287111 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.287734 23281 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.292394 23475 raft_consensus.cc:2804] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.293251 23475 raft_consensus.cc:493] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.293680 23475 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.297451 23281 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 1.
I20260709 12:12:14.298648 23394 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.299537 23486 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.301769 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
W20260709 12:12:14.303371 23394 leader_election.cc:343] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.304539 23480 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.317040 23476 raft_consensus.cc:493] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.317027 23486 raft_consensus.cc:697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:14.317894 23476 raft_consensus.cc:515] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.318672 23486 consensus_queue.cc:237] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.320504 23476 leader_election.cc:290] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.321717 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.324481 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
W20260709 12:12:14.326959 23245 leader_election.cc:343] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:14.328001 23246 leader_election.cc:343] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.328464 23246 leader_election.cc:304] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741; no voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.329655 23476 raft_consensus.cc:2749] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.332485 23475 raft_consensus.cc:515] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.338223 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
W20260709 12:12:14.340047 23245 leader_election.cc:343] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.342927 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.343545 23430 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.351861 23430 raft_consensus.cc:2468] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.353655 23246 leader_election.cc:304] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: b4d5867a2db442d291806a1ac0039588
I20260709 12:12:14.354840 23476 raft_consensus.cc:2804] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.356454 23476 raft_consensus.cc:697] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:14.357343 23476 consensus_queue.cc:237] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.359123 23482 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.359699 23482 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.361966 23482 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.364463 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.365445 23429 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 0.
I20260709 12:12:14.367103 23321 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.371428 23482 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.371613 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.372854 23482 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.373407 23482 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:14.380479 23320 leader_election.cc:343] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.387374 23176 catalog_manager.cc:5697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 1 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:14.392302 23475 leader_election.cc:290] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.413072 23482 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.413008 23177 catalog_manager.cc:5697] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.417434 23474 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.417943 23474 ts_tablet_manager.cc:1403] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.169s	user 0.058s	sys 0.101s
I20260709 12:12:14.418416 23471 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.418902 23471 ts_tablet_manager.cc:1403] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.165s	user 0.060s	sys 0.097s
I20260709 12:12:14.419885 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.420960 23474 raft_consensus.cc:359] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.421823 23474 raft_consensus.cc:385] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:12:14.421852 23320 leader_election.cc:343] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.422291 23474 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.421873 23471 raft_consensus.cc:359] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.422760 23471 raft_consensus.cc:385] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.423064 23471 raft_consensus.cc:740] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.423172 23474 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.423936 23471 consensus_queue.cc:260] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.426342 23471 ts_tablet_manager.cc:1434] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:12:14.416882 23470 tablet_bootstrap.cc:492] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.427152 23470 ts_tablet_manager.cc:1403] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.180s	user 0.057s	sys 0.062s
I20260709 12:12:14.427351 23471 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.429952 23470 raft_consensus.cc:359] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.424544 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.430714 23470 raft_consensus.cc:385] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.431046 23470 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.431116 23430 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.432004 23470 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.432621 23474 ts_tablet_manager.cc:1434] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.014s	user 0.014s	sys 0.001s
I20260709 12:12:14.434358 23471 tablet_bootstrap.cc:654] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.437269 23470 ts_tablet_manager.cc:1434] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:12:14.437865 23474 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.438216 23470 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.444878 23470 tablet_bootstrap.cc:654] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.445830 23471 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.446318 23471 ts_tablet_manager.cc:1403] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:12:14.449259 23471 raft_consensus.cc:359] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.450027 23471 raft_consensus.cc:385] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.450304 23471 raft_consensus.cc:740] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.451213 23471 consensus_queue.cc:260] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.452979 23474 tablet_bootstrap.cc:654] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.453783 23471 ts_tablet_manager.cc:1434] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:14.454854 23471 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.457149 23486 raft_consensus.cc:493] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.457826 23486 raft_consensus.cc:515] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.461416 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.461987 23471 tablet_bootstrap.cc:654] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.462244 23281 raft_consensus.cc:2468] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 0.
I20260709 12:12:14.464483 23394 leader_election.cc:304] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.464888 23474 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.465516 23474 ts_tablet_manager.cc:1403] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.007s
I20260709 12:12:14.466418 23500 raft_consensus.cc:2804] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.466895 23500 raft_consensus.cc:493] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.467242 23500 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.468842 23474 raft_consensus.cc:359] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.469817 23474 raft_consensus.cc:385] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.470124 23474 raft_consensus.cc:740] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.470916 23474 consensus_queue.cc:260] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.472555 23476 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.473112 23476 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.473583 23471 tablet_bootstrap.cc:492] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.474203 23471 ts_tablet_manager.cc:1403] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.012s
I20260709 12:12:14.476989 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.477197 23471 raft_consensus.cc:359] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.477759 23355 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.478073 23471 raft_consensus.cc:385] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.478408 23471 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.479229 23471 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.480815 23474 ts_tablet_manager.cc:1434] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:12:14.480777 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.481735 23471 ts_tablet_manager.cc:1434] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.001s
I20260709 12:12:14.481627 23429 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.482872 23471 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.475356 23476 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.479199 23245 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.486240 23474 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.490110 23471 tablet_bootstrap.cc:654] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.491025 23476 raft_consensus.cc:2804] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.491479 23476 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.491878 23476 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.494278 23474 tablet_bootstrap.cc:654] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.504410 23471 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.504904 23471 ts_tablet_manager.cc:1403] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 12:12:14.505877 23474 tablet_bootstrap.cc:492] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.506350 23474 ts_tablet_manager.cc:1403] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:12:14.507422 23471 raft_consensus.cc:359] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.508127 23471 raft_consensus.cc:385] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.508391 23471 raft_consensus.cc:740] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.509250 23474 raft_consensus.cc:359] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.509294 23471 consensus_queue.cc:260] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.510021 23474 raft_consensus.cc:385] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.510324 23474 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.511237 23474 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.511837 23471 ts_tablet_manager.cc:1434] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:12:14.512755 23471 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
I20260709 12:12:14.519635 23471 tablet_bootstrap.cc:654] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.519932 23482 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.520553 23474 ts_tablet_manager.cc:1434] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:12:14.524086 23471 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.524395 23474 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.524526 23471 ts_tablet_manager.cc:1403] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:12:14.527117 23471 raft_consensus.cc:359] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.527858 23471 raft_consensus.cc:385] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.528203 23471 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.529049 23471 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.524979 23500 raft_consensus.cc:515] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.523087 23486 leader_election.cc:290] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.535138 23474 tablet_bootstrap.cc:654] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.536299 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.537004 23281 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.539009 23474 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.539351 23474 ts_tablet_manager.cc:1403] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:12:14.531317 23471 ts_tablet_manager.cc:1434] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:14.522513 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.540529 23471 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: Bootstrap starting.
W20260709 12:12:14.540884 23394 leader_election.cc:343] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.543776 23474 raft_consensus.cc:359] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.544499 23474 raft_consensus.cc:385] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.544773 23474 raft_consensus.cc:740] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.545128 23281 raft_consensus.cc:2468] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 1.
I20260709 12:12:14.546299 23394 leader_election.cc:304] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:14.532445 23476 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.547693 23430 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 1.
I20260709 12:12:14.548309 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:14.549548 23321 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.546679 23471 tablet_bootstrap.cc:654] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.550395 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:14.551002 23482 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:12:14.551316 23394 leader_election.cc:343] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.551013 23355 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.551994 23470 tablet_bootstrap.cc:492] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.552500 23470 ts_tablet_manager.cc:1403] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.115s	user 0.040s	sys 0.069s
I20260709 12:12:14.552757 23475 raft_consensus.cc:493] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.553197 23475 raft_consensus.cc:515] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.555246 23475 leader_election.cc:290] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.555114 23470 raft_consensus.cc:359] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.555809 23470 raft_consensus.cc:385] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.556085 23470 raft_consensus.cc:740] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.556867 23470 consensus_queue.cc:260] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.557428 23480 raft_consensus.cc:2804] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.558068 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.559192 23470 ts_tablet_manager.cc:1434] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:14.560118 23470 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
W20260709 12:12:14.561987 23246 leader_election.cc:343] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.565786 23470 tablet_bootstrap.cc:654] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.566174 23355 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.567942 23245 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.568976 23502 raft_consensus.cc:2804] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.570778 23486 consensus_queue.cc:1048] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:14.545553 23474 consensus_queue.cc:260] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.582221 23482 raft_consensus.cc:697] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:14.584091 23482 consensus_queue.cc:237] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.546618 23500 leader_election.cc:290] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:14.585381 23474 ts_tablet_manager.cc:1434] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:12:14.597229 23474 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.589569 23480 raft_consensus.cc:697] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:14.600129 23480 consensus_queue.cc:237] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.601730 23504 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.602231 23504 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.600096 23355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.604339 23355 raft_consensus.cc:2468] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.604512 23504 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.605851 23245 leader_election.cc:304] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.606856 23476 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.607148 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.607537 23476 raft_consensus.cc:2804] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.607942 23476 raft_consensus.cc:493] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.608170 23281 raft_consensus.cc:2393] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4d5867a2db442d291806a1ac0039588 in current term 1: Already voted for candidate bff7757456664d9a86c2c2b9bb12dcb3 in this term.
I20260709 12:12:14.608282 23476 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.607267 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.610054 23474 tablet_bootstrap.cc:654] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.608376 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.610896 23429 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.611907 23504 raft_consensus.cc:493] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.612396 23504 raft_consensus.cc:515] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.614554 23504 leader_election.cc:290] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.610127 23321 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4d5867a2db442d291806a1ac0039588; no voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.616264 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
W20260709 12:12:14.618006 23320 leader_election.cc:343] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.619158 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.619938 23430 raft_consensus.cc:2468] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 0.
I20260709 12:12:14.621768 23321 leader_election.cc:304] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.622987 23504 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.627411 23474 tablet_bootstrap.cc:492] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.636520 23474 ts_tablet_manager.cc:1403] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.013s
I20260709 12:12:14.637043 23482 raft_consensus.cc:2804] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.637495 23482 raft_consensus.cc:493] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.637903 23482 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.634955 23429 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.638711 23476 raft_consensus.cc:515] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.645777 23502 raft_consensus.cc:697] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:14.646291 23470 tablet_bootstrap.cc:492] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.646729 23470 ts_tablet_manager.cc:1403] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.087s	user 0.019s	sys 0.008s
I20260709 12:12:14.641314 23176 catalog_manager.cc:5697] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.649377 23470 raft_consensus.cc:359] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.650051 23470 raft_consensus.cc:385] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.650470 23470 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.647807 23474 raft_consensus.cc:359] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.646894 23502 consensus_queue.cc:237] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.651015 23471 tablet_bootstrap.cc:492] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: No bootstrap required, opened a new log
I20260709 12:12:14.651145 23470 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.651476 23486 consensus_queue.cc:1048] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:14.652841 23470 ts_tablet_manager.cc:1434] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:12:14.653724 23470 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: Bootstrap starting.
I20260709 12:12:14.653769 23476 leader_election.cc:290] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.655990 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:14.656682 23354 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.651507 23471 ts_tablet_manager.cc:1403] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent bootstrapping tablet: real 0.111s	user 0.043s	sys 0.042s
I20260709 12:12:14.650929 23474 raft_consensus.cc:385] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.662671 23474 raft_consensus.cc:740] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.662884 23471 raft_consensus.cc:359] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.664322 23474 consensus_queue.cc:260] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.660121 23504 raft_consensus.cc:2749] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.666131 23471 raft_consensus.cc:385] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.671120 23471 raft_consensus.cc:740] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.660004 23470 tablet_bootstrap.cc:654] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.668126 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.668023 23178 catalog_manager.cc:5697] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 1 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:14.671954 23471 consensus_queue.cc:260] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.676175 23471 ts_tablet_manager.cc:1434] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:12:14.677052 23429 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.684374 23176 catalog_manager.cc:5697] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.668929 23482 raft_consensus.cc:515] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.686968 23474 ts_tablet_manager.cc:1434] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:12:14.672592 23354 raft_consensus.cc:2468] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.688659 23474 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: Bootstrap starting.
I20260709 12:12:14.689121 23245 leader_election.cc:304] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.689746 23280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.690256 23502 raft_consensus.cc:2804] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.691064 23502 raft_consensus.cc:697] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
W20260709 12:12:14.691247 23320 leader_election.cc:343] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:14.692073 23502 consensus_queue.cc:237] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.693467 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.694073 23430 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.694849 23482 leader_election.cc:290] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.702039 23429 raft_consensus.cc:2468] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.722770 23470 tablet_bootstrap.cc:492] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: No bootstrap required, opened a new log
I20260709 12:12:14.723268 23470 ts_tablet_manager.cc:1403] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: Time spent bootstrapping tablet: real 0.070s	user 0.021s	sys 0.008s
I20260709 12:12:14.720700 23430 raft_consensus.cc:2468] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 1.
I20260709 12:12:14.725180 23482 raft_consensus.cc:493] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.725768 23482 raft_consensus.cc:515] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.726938 23178 catalog_manager.cc:5697] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.728099 23482 leader_election.cc:290] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.728413 23474 tablet_bootstrap.cc:654] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:14.729904 23280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.730541 23280 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 0.
I20260709 12:12:14.732403 23321 leader_election.cc:304] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.733295 23526 raft_consensus.cc:2804] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.733780 23526 raft_consensus.cc:697] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:14.734025 23320 leader_election.cc:304] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.734990 23524 raft_consensus.cc:2804] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.734617 23526 consensus_queue.cc:237] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.735349 23524 raft_consensus.cc:493] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.735709 23524 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.737687 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.738413 23430 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 0.
I20260709 12:12:14.746069 23524 raft_consensus.cc:515] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.749300 23470 raft_consensus.cc:359] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.749910 23474 tablet_bootstrap.cc:492] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: No bootstrap required, opened a new log
I20260709 12:12:14.750411 23474 ts_tablet_manager.cc:1403] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: Time spent bootstrapping tablet: real 0.062s	user 0.006s	sys 0.007s
I20260709 12:12:14.751772 23280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:14.752363 23280 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.753342 23474 raft_consensus.cc:359] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.753993 23474 raft_consensus.cc:385] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.754264 23474 raft_consensus.cc:740] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.755110 23474 consensus_queue.cc:260] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.755797 23524 leader_election.cc:290] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.756287 23470 raft_consensus.cc:385] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:14.756593 23470 raft_consensus.cc:740] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Initialized, Role: FOLLOWER
I20260709 12:12:14.757318 23474 ts_tablet_manager.cc:1434] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:14.757236 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.757917 23430 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.759496 23280 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 1.
I20260709 12:12:14.757251 23470 consensus_queue.cc:260] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.760637 23320 leader_election.cc:304] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.761495 23524 raft_consensus.cc:2804] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.762144 23524 raft_consensus.cc:697] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:14.762914 23470 ts_tablet_manager.cc:1434] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.001s
I20260709 12:12:14.763060 23524 consensus_queue.cc:237] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.760356 23178 catalog_manager.cc:5697] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.766496 23430 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 1.
I20260709 12:12:14.775843 23524 raft_consensus.cc:493] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.776332 23524 raft_consensus.cc:515] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.779096 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:14.780503 23524 leader_election.cc:290] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.780038 23176 catalog_manager.cc:5697] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.780591 23480 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:14.781059 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.781893 23429 raft_consensus.cc:2393] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4d5867a2db442d291806a1ac0039588 in current term 1: Already voted for candidate 95f0c264b4dc4fe28d874331ff366741 in this term.
I20260709 12:12:14.783088 23321 leader_election.cc:304] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4d5867a2db442d291806a1ac0039588; no voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.784587 23524 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.787089 23502 consensus_queue.cc:1048] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:14.796962 23524 raft_consensus.cc:2749] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.800891 23486 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:14.805913 23516 consensus_queue.cc:1048] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:14.826581 23516 raft_consensus.cc:493] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.827013 23516 raft_consensus.cc:515] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.828631 23516 leader_election.cc:290] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.829487 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.829756 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.830014 23356 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.830386 23430 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 0.
I20260709 12:12:14.830953 23245 leader_election.cc:304] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.831689 23516 raft_consensus.cc:2804] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:14.832008 23516 raft_consensus.cc:493] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.832228 23516 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.837035 23516 raft_consensus.cc:515] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.838788 23516 leader_election.cc:290] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.839571 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:14.839866 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:14.840091 23356 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.840395 23430 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.844882 23356 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.844882 23430 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 1.
I20260709 12:12:14.846170 23245 leader_election.cc:304] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:14.846819 23516 raft_consensus.cc:2804] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:14.847174 23516 raft_consensus.cc:697] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:14.847960 23516 consensus_queue.cc:237] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.855352 23176 catalog_manager.cc:5697] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:14.918624 23516 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.919054 23516 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:14.920754 23516 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:14.921631 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:14.921805 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:14.922461 23430 raft_consensus.cc:2393] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95f0c264b4dc4fe28d874331ff366741 in current term 1: Already voted for candidate b4d5867a2db442d291806a1ac0039588 in this term.
I20260709 12:12:14.923399 23246 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741; no voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:14.924058 23516 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:14.928843 23516 raft_consensus.cc:2749] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:14.955811 23430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:51032
I20260709 12:12:14.956382 23430 raft_consensus.cc:606] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.961688 23430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:51032
I20260709 12:12:14.962231 23430 raft_consensus.cc:606] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.976104 23356 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:54600
I20260709 12:12:14.976624 23356 raft_consensus.cc:606] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.982663 23430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:51032
I20260709 12:12:14.983173 23430 raft_consensus.cc:606] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.988827 23356 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4908b7f19700429584ac45090f98580d"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:54600
I20260709 12:12:14.989380 23356 raft_consensus.cc:606] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:14.996039 23486 consensus_queue.cc:1048] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.004508 23514 raft_consensus.cc:993] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.005002 23514 raft_consensus.cc:1081] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.012337 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56260
I20260709 12:12:15.013002 23281 raft_consensus.cc:493] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.013531 23281 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.016007 23356 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:15.020452 23514 raft_consensus.cc:993] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.020120 23281 raft_consensus.cc:515] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.020915 23480 raft_consensus.cc:1081] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.021391 23280 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:56274
I20260709 12:12:15.021998 23280 raft_consensus.cc:606] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Received request to transfer leadership to TS b4d5867a2db442d291806a1ac0039588
I20260709 12:12:15.022439 23281 leader_election.cc:290] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.023320 23279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56260
I20260709 12:12:15.023765 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.023934 23279 raft_consensus.cc:493] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.024376 23430 raft_consensus.cc:3055] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.024544 23279 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.024765 23430 raft_consensus.cc:740] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:15.024547 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.025830 23430 consensus_queue.cc:260] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.027025 23430 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.032226 23526 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.033051 23524 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.034273 23430 raft_consensus.cc:2468] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.034574 23279 raft_consensus.cc:515] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:15.035826 23246 leader_election.cc:304] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: b4d5867a2db442d291806a1ac0039588
I20260709 12:12:15.037184 23279 leader_election.cc:290] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:15.037127 23517 raft_consensus.cc:2804] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.040972 23517 raft_consensus.cc:697] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.041981 23517 consensus_queue.cc:237] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.049504 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.051957 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ae90aa7d14a48a4ace7673c6f365c06" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.052703 23430 raft_consensus.cc:3055] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.053056 23430 raft_consensus.cc:740] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:15.053999 23430 consensus_queue.cc:260] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:15.055476 23430 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.050174 23354 raft_consensus.cc:3060] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.063740 23279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:56274
I20260709 12:12:15.064468 23279 raft_consensus.cc:606] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Received request to transfer leadership to TS b4d5867a2db442d291806a1ac0039588
I20260709 12:12:15.065131 23354 raft_consensus.cc:2468] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.063540 23430 raft_consensus.cc:2468] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.067207 23245 leader_election.cc:304] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:15.068044 23517 raft_consensus.cc:2804] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.068574 23517 raft_consensus.cc:697] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.069362 23517 consensus_queue.cc:237] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } }
I20260709 12:12:15.071242 23504 raft_consensus.cc:493] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.071724 23504 raft_consensus.cc:515] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.073868 23482 raft_consensus.cc:993] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.073812 23504 leader_election.cc:290] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.074337 23482 raft_consensus.cc:1081] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.079473 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:15.079711 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cde435c4ef4b40b4c2c179b305b37e" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95f0c264b4dc4fe28d874331ff366741" is_pre_election: true
I20260709 12:12:15.080359 23429 raft_consensus.cc:2393] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4d5867a2db442d291806a1ac0039588 in current term 2: Already voted for candidate 95f0c264b4dc4fe28d874331ff366741 in this term.
I20260709 12:12:15.082293 23279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56264
I20260709 12:12:15.082163 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:56274
I20260709 12:12:15.083035 23320 leader_election.cc:304] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4d5867a2db442d291806a1ac0039588; no voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:15.083153 23281 raft_consensus.cc:606] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Received request to transfer leadership to TS b4d5867a2db442d291806a1ac0039588
I20260709 12:12:15.083761 23482 raft_consensus.cc:3060] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.084149 23279 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.084510 23279 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.088721 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:56274
I20260709 12:12:15.089243 23281 raft_consensus.cc:606] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Received request to transfer leadership to TS bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:15.090766 23482 raft_consensus.cc:2749] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:15.091212 23279 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.093398 23279 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.094018 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.094183 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.094548 23429 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.094739 23354 raft_consensus.cc:3055] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.095090 23354 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:15.095762 23354 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.096673 23354 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.097790 23176 catalog_manager.cc:5697] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.099315 23176 catalog_manager.cc:5697] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.100822 23516 raft_consensus.cc:493] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.101240 23429 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.101500 23354 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.101295 23516 raft_consensus.cc:515] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.102530 23245 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:15.103152 23517 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.103361 23516 leader_election.cc:290] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.103698 23517 raft_consensus.cc:697] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.104331 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d5867a2db442d291806a1ac0039588" is_pre_election: true
I20260709 12:12:15.104425 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" is_pre_election: true
I20260709 12:12:15.104565 23517 consensus_queue.cc:237] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.105281 23429 raft_consensus.cc:2393] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95f0c264b4dc4fe28d874331ff366741 in current term 1: Already voted for candidate b4d5867a2db442d291806a1ac0039588 in this term.
I20260709 12:12:15.106333 23246 leader_election.cc:304] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741; no voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:15.107124 23502 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:15.114158 23502 raft_consensus.cc:2749] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:15.117304 23176 catalog_manager.cc:5697] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from b4d5867a2db442d291806a1ac0039588 (127.22.25.66) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.186573 23502 consensus_queue.cc:1048] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.188097 23502 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.211866 23517 consensus_queue.cc:1048] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:15.245395 23482 consensus_queue.cc:1048] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.248422 23482 consensus_queue.cc:1048] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:15.253171 23517 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.260108 23502 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.270098 23502 raft_consensus.cc:993] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:15.271387 23476 raft_consensus.cc:1081] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Signalling peer bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:15.289731 23476 raft_consensus.cc:993] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.290167 23502 raft_consensus.cc:1081] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Signalling peer b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.290850 23428 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:51000
I20260709 12:12:15.291559 23428 raft_consensus.cc:493] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.291965 23428 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.292260 23354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:54584
I20260709 12:12:15.292821 23354 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.293183 23354 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.295177 23482 consensus_queue.cc:1048] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.299373 23428 raft_consensus.cc:515] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.300248 23354 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.302994 23354 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.305657 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:15.306111 23281 raft_consensus.cc:3055] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.306317 23281 raft_consensus.cc:740] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.306977 23281 consensus_queue.cc:260] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.308229 23428 leader_election.cc:290] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 2 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:15.308822 23502 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.309109 23281 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.309238 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.309970 23430 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.310829 23356 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a841b5028c46c0927d865e58b0f3c9" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.311400 23356 raft_consensus.cc:3060] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.312559 23280 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:15.313145 23280 raft_consensus.cc:3055] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.313458 23280 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.314224 23280 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.315482 23280 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.320247 23356 raft_consensus.cc:2468] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 2.
I20260709 12:12:15.321759 23394 leader_election.cc:304] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:15.322654 23500 raft_consensus.cc:2804] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.323127 23500 raft_consensus.cc:697] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:15.323968 23500 consensus_queue.cc:237] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.329831 23430 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.331246 23321 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:15.332218 23482 raft_consensus.cc:2804] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.332770 23482 raft_consensus.cc:697] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:15.333648 23482 consensus_queue.cc:237] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.334002 23177 catalog_manager.cc:5697] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 1 to 2, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 2 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:15.337606 23280 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.338111 23516 raft_consensus.cc:993] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.327790 23281 raft_consensus.cc:2468] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 2.
I20260709 12:12:15.338590 23516 raft_consensus.cc:1081] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Signalling peer b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.343030 23504 raft_consensus.cc:993] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.343498 23504 raft_consensus.cc:1081] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.348803 23354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:54584
I20260709 12:12:15.351924 23279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4908b7f19700429584ac45090f98580d"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56264
I20260709 12:12:15.352149 23354 raft_consensus.cc:493] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.352635 23354 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.353668 23178 catalog_manager.cc:5697] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 1 to 2, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.355877 23526 consensus_queue.cc:1048] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.359475 23279 raft_consensus.cc:493] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.360442 23279 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.366505 23354 raft_consensus.cc:515] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.368719 23354 leader_election.cc:290] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.369230 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:15.369815 23281 raft_consensus.cc:3055] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.370121 23281 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.370798 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.370918 23281 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.370258 23279 raft_consensus.cc:515] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.372279 23281 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.371372 23430 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.374432 23279 leader_election.cc:290] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.375362 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.375442 23429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4908b7f19700429584ac45090f98580d" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.376935 23354 raft_consensus.cc:3055] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.377302 23354 raft_consensus.cc:740] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:15.378161 23354 consensus_queue.cc:260] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.379667 23354 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.382792 23281 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.384146 23320 leader_election.cc:304] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:15.384590 23430 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.385054 23482 raft_consensus.cc:2804] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.385537 23482 raft_consensus.cc:697] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:15.386349 23482 consensus_queue.cc:237] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.387110 23354 raft_consensus.cc:2468] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.388510 23245 leader_election.cc:304] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:15.389252 23502 raft_consensus.cc:2804] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.389676 23502 raft_consensus.cc:697] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.390448 23502 consensus_queue.cc:237] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.393339 23178 catalog_manager.cc:5697] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 1 to 2, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.394239 23516 raft_consensus.cc:993] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.394649 23516 raft_consensus.cc:1081] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Signalling peer b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:15.396306 23354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:54584
I20260709 12:12:15.396988 23354 raft_consensus.cc:493] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.397290 23354 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.397718 23179 catalog_manager.cc:5697] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from b4d5867a2db442d291806a1ac0039588 (127.22.25.66) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.402635 23354 raft_consensus.cc:515] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.404855 23354 leader_election.cc:290] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.406180 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:15.406771 23281 raft_consensus.cc:3055] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.407100 23281 raft_consensus.cc:740] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.407428 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf956843dd6547a19fd10e4fd69dc0bb" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.408049 23430 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.408154 23281 consensus_queue.cc:260] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.409409 23281 raft_consensus.cc:3060] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.410887 23500 raft_consensus.cc:993] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.411290 23500 raft_consensus.cc:1081] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:15.412746 23279 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56260
I20260709 12:12:15.413199 23279 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:15.413550 23279 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.415776 23430 raft_consensus.cc:2468] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.417047 23321 leader_election.cc:304] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:15.417768 23482 raft_consensus.cc:2804] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.418161 23482 raft_consensus.cc:697] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:15.418301 23281 raft_consensus.cc:2468] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 2.
I20260709 12:12:15.419008 23482 consensus_queue.cc:237] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.420636 23279 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.423389 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:15.423985 23354 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.424220 23279 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:15.425254 23430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:15.425854 23430 raft_consensus.cc:3055] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:15.426192 23430 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:15.426915 23430 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.427839 23430 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.429946 23354 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.430959 23245 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:15.430143 23179 catalog_manager.cc:5697] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 1 to 2, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.431667 23517 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:15.432060 23517 raft_consensus.cc:697] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:15.432799 23517 consensus_queue.cc:237] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:15.435314 23430 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:15.440222 23178 catalog_manager.cc:5697] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:15.500916 23354 raft_consensus.cc:1275] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.502517 23517 consensus_queue.cc:1048] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.511929 23430 raft_consensus.cc:1275] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.512753 23428 raft_consensus.cc:1275] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.513911 23502 consensus_queue.cc:1048] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.514786 23476 consensus_queue.cc:1048] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.526441 23354 raft_consensus.cc:1275] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.528147 23502 consensus_queue.cc:1048] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:15.535610 23356 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.537411 23502 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.546300 23430 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.547902 23502 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.734758 23279 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.736362 23482 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.737076 23354 raft_consensus.cc:1275] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.738360 23500 consensus_queue.cc:1048] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.747205 23430 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.748689 23279 raft_consensus.cc:1275] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.750240 23480 consensus_queue.cc:1048] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.750442 23504 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.801487 23279 raft_consensus.cc:1275] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.802747 23504 consensus_queue.cc:1048] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.811066 23279 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.812999 23526 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.814924 23430 raft_consensus.cc:1275] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.816457 23504 consensus_queue.cc:1048] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.822744 23428 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.824311 23524 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.974710 23428 raft_consensus.cc:3060] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:15.979800 23428 raft_consensus.cc:1275] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.981268 23476 consensus_queue.cc:1048] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:15.988960 23354 raft_consensus.cc:1275] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:15.990362 23502 consensus_queue.cc:1048] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:16.041694 23354 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:16.043411 23502 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:16.046799 23430 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:16.048382 23476 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:18.126026 22629 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:12:18.126678 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:18.143175 23279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:18.143894 23279 raft_consensus.cc:606] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Received request to transfer leadership to TS bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:18.146879 23279 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:18.147537 23279 raft_consensus.cc:606] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Received request to transfer leadership to TS bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:18.148728 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:12:18.149173 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:18.331960 23565 raft_consensus.cc:993] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:18.332394 23565 raft_consensus.cc:1081] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Signalling peer bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:18.333866 23430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:51000
I20260709 12:12:18.334317 23430 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:18.334586 23430 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.338809 23430 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.340219 23430 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 3 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:18.341275 23279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:18.341764 23279 raft_consensus.cc:3055] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:18.341742 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:18.342149 23279 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:18.342330 23354 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.342857 23279 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.343791 23279 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.347323 23354 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 3.
I20260709 12:12:18.348583 23279 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 3.
I20260709 12:12:18.348608 23394 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:18.349671 23562 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:18.350673 23562 raft_consensus.cc:697] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:18.351354 23562 consensus_queue.cc:237] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.359135 23177 catalog_manager.cc:5697] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 2 to 3, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 3 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:18.648264 23565 raft_consensus.cc:993] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:18.648706 23565 raft_consensus.cc:1081] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Signalling peer bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:18.649973 23430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:51000
I20260709 12:12:18.650388 23430 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:18.650625 23430 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.654932 23430 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.656391 23430 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 3 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:18.657661 23279 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:18.658437 23279 raft_consensus.cc:3055] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:18.658334 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:18.658882 23279 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:18.659096 23354 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.659869 23279 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.661253 23279 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:18.666220 23354 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 3.
I20260709 12:12:18.667690 23394 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:18.667784 23279 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 3.
I20260709 12:12:18.668568 23562 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:18.669166 23562 raft_consensus.cc:697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:18.669893 23562 consensus_queue.cc:237] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:18.680006 23177 catalog_manager.cc:5697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 2 to 3, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 3 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:18.918226 23354 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:18.919852 23562 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:18.930263 23279 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:18.932346 23562 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.150107 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:19.150298 23279 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:19.151551 23562 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:19.158344 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:19.158977 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:19.165539 23354 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:19.167157 23582 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:20.160466 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:20.165941 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:20.166373 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:21.167341 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:21.172861 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:21.173430 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:22.174415 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:22.180444 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:22.180840 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:23.182173 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:23.190815 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:23.191269 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:24.192399 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:24.198773 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:24.199404 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:25.200351 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:25.208228 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:25.208748 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:26.209724 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:26.216780 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:26.217228 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:27.218291 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.225085 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:27.225538 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:28.257917 22629 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:12:28.266075 23428 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:51032
I20260709 12:12:28.266750 23428 raft_consensus.cc:606] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:28.272511 23428 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:51032
I20260709 12:12:28.273018 23428 raft_consensus.cc:606] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:28.285090 23354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:54600
I20260709 12:12:28.285626 23354 raft_consensus.cc:606] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:28.291198 23354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:54600
I20260709 12:12:28.291797 23354 raft_consensus.cc:606] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:28.296911 23354 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
mode: GRACEFUL
new_leader_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:54600
I20260709 12:12:28.297478 23354 raft_consensus.cc:606] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Received request to transfer leadership to TS 95f0c264b4dc4fe28d874331ff366741
I20260709 12:12:28.299850 23595 raft_consensus.cc:993] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.300444 23591 raft_consensus.cc:1081] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.302069 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56260
I20260709 12:12:28.302583 23281 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.302935 23281 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.310153 23281 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.313714 23281 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 4 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:28.314244 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:28.314742 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:28.315127 23354 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.315687 23428 raft_consensus.cc:3055] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:28.316045 23428 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:28.317162 23428 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.318539 23428 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.322947 23354 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 4.
I20260709 12:12:28.324563 23245 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:28.325567 23565 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:28.326239 23428 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 4.
I20260709 12:12:28.326808 23565 raft_consensus.cc:697] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:28.327877 23565 consensus_queue.cc:237] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.335386 23179 catalog_manager.cc:5697] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 3 to 4, leader changed from bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.381445 23594 raft_consensus.cc:993] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.382164 23593 raft_consensus.cc:1081] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.383996 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56264
I20260709 12:12:28.384610 23281 raft_consensus.cc:493] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.385046 23281 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.390059 23281 raft_consensus.cc:515] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.391759 23281 leader_election.cc:290] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 3 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:28.392629 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:28.392711 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:28.393296 23354 raft_consensus.cc:3055] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:28.393407 23428 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.393651 23354 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:28.394485 23354 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.395449 23354 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.399986 23354 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 3.
I20260709 12:12:28.400974 23245 leader_election.cc:304] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:28.401638 23565 raft_consensus.cc:2804] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:28.402010 23565 raft_consensus.cc:697] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:28.402431 23428 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 3.
I20260709 12:12:28.402729 23565 consensus_queue.cc:237] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.410290 23179 catalog_manager.cc:5697] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 2 to 3, leader changed from b4d5867a2db442d291806a1ac0039588 (127.22.25.66) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.425143 23594 raft_consensus.cc:993] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.425596 23593 raft_consensus.cc:1081] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.427037 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56264
I20260709 12:12:28.427451 23281 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.427752 23281 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.431903 23281 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.433415 23281 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 3 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:28.434268 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:28.434510 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:28.434862 23354 raft_consensus.cc:3055] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:28.435031 23428 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.435174 23354 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:28.436172 23354 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.437613 23354 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:28.442857 23428 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 3.
I20260709 12:12:28.443955 23354 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 3.
I20260709 12:12:28.444411 23246 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:28.445144 23565 raft_consensus.cc:2804] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:28.445475 23565 raft_consensus.cc:697] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:28.446201 23565 consensus_queue.cc:237] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.452369 23179 catalog_manager.cc:5697] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 2 to 3, leader changed from b4d5867a2db442d291806a1ac0039588 (127.22.25.66) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.453438 23594 raft_consensus.cc:993] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.453912 23593 raft_consensus.cc:1081] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.455657 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56264
I20260709 12:12:28.456228 23281 raft_consensus.cc:493] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.456632 23281 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.463730 23281 raft_consensus.cc:515] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.466166 23281 leader_election.cc:290] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:28.467031 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:28.467334 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60afbcca2b847fe93f7451e37f98b79" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:28.467681 23354 raft_consensus.cc:3055] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:28.467881 23428 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.468006 23354 raft_consensus.cc:740] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:28.468902 23354 consensus_queue.cc:260] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.470100 23354 raft_consensus.cc:3060] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:28.475144 23428 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:28.476374 23354 raft_consensus.cc:2468] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 2.
I20260709 12:12:28.476436 23246 leader_election.cc:304] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:28.477387 23565 raft_consensus.cc:2804] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:28.477872 23565 raft_consensus.cc:697] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:28.478559 23565 consensus_queue.cc:237] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.486330 23179 catalog_manager.cc:5697] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 1 to 2, leader changed from b4d5867a2db442d291806a1ac0039588 (127.22.25.66) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.657852 23595 raft_consensus.cc:993] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: : Instructing follower 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.658320 23595 raft_consensus.cc:1081] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Signalling peer 95f0c264b4dc4fe28d874331ff366741 to start an election
I20260709 12:12:28.660197 23281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
 from {username='slave'} at 127.0.0.1:56260
I20260709 12:12:28.660777 23281 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.661080 23281 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.666714 23281 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.668368 23281 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 4 election: Requested vote from peers b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:28.669301 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:28.669450 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "95f0c264b4dc4fe28d874331ff366741" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:28.669922 23354 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.669989 23428 raft_consensus.cc:3055] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:28.670358 23428 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:28.671231 23428 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.672609 23428 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:28.676848 23354 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 4.
I20260709 12:12:28.678572 23245 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f0c264b4dc4fe28d874331ff366741, b4d5867a2db442d291806a1ac0039588; no voters: 
I20260709 12:12:28.679430 23565 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:28.680212 23565 raft_consensus.cc:697] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Becoming Leader. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:28.682442 23428 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f0c264b4dc4fe28d874331ff366741 in term 4.
I20260709 12:12:28.682256 23565 consensus_queue.cc:237] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:28.688849 23179 catalog_manager.cc:5697] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 reported cstate change: term changed from 3 to 4, leader changed from bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67) to 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "95f0c264b4dc4fe28d874331ff366741" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:28.865459 23428 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:28.866968 23565 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.871147 23354 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:28.872722 23565 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:28.924131 23428 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:28.925421 23607 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:28.934432 23354 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:28.937016 23565 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:28.944012 23428 raft_consensus.cc:1275] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.945222 23565 consensus_queue.cc:1048] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:28.956728 23354 raft_consensus.cc:1275] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:28.958909 23607 consensus_queue.cc:1048] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:29.007445 23428 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:29.008746 23607 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:29.020498 23354 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:29.021981 23615 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:29.228497 23428 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:29.229737 23615 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:29.238245 23354 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Refusing update from remote peer 95f0c264b4dc4fe28d874331ff366741: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:29.239871 23615 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:31.336082 22629 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:12:31.336733 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:31.344856 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:31.345293 23281 raft_consensus.cc:606] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Received request to transfer leadership to TS bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:31.347438 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:31.347872 23281 raft_consensus.cc:606] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Received request to transfer leadership to TS b4d5867a2db442d291806a1ac0039588
I20260709 12:12:31.349846 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:31.350240 23281 raft_consensus.cc:606] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Received request to transfer leadership to TS b4d5867a2db442d291806a1ac0039588
I20260709 12:12:31.352159 23281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
mode: GRACEFUL
new_leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:60394
I20260709 12:12:31.352555 23281 raft_consensus.cc:606] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Received request to transfer leadership to TS bff7757456664d9a86c2c2b9bb12dcb3
I20260709 12:12:31.353471 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:31.354048 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:31.367434 23615 raft_consensus.cc:993] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:31.367919 23622 raft_consensus.cc:1081] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Signalling peer b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:31.369225 23354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:54584
I20260709 12:12:31.369633 23354 raft_consensus.cc:493] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:31.369884 23354 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.374047 23354 raft_consensus.cc:515] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.375883 23354 leader_election.cc:290] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 4 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:31.376593 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:31.376878 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a697b070e4cf4d599a7959216dafa9b7" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:31.377118 23281 raft_consensus.cc:3055] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:31.377332 23428 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.377416 23281 raft_consensus.cc:740] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:31.378105 23281 consensus_queue.cc:260] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.378912 23281 raft_consensus.cc:3060] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.381557 23428 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 4.
I20260709 12:12:31.382635 23321 leader_election.cc:304] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:31.383246 23593 raft_consensus.cc:2804] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:31.383725 23281 raft_consensus.cc:2468] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 4.
I20260709 12:12:31.384289 23593 raft_consensus.cc:697] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 4 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:31.385124 23593 consensus_queue.cc:237] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.391652 23178 catalog_manager.cc:5697] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 3 to 4, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:31.458472 23615 raft_consensus.cc:993] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:31.458907 23622 raft_consensus.cc:1081] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Signalling peer bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:31.460284 23428 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:51000
I20260709 12:12:31.460721 23428 raft_consensus.cc:493] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:31.460956 23428 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.465198 23428 raft_consensus.cc:515] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.467085 23428 leader_election.cc:290] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 5 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:31.468139 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:31.468230 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03866111a0a54ab2ad035e9c53d7c6ce" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:31.468607 23281 raft_consensus.cc:3055] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:31.468662 23354 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.468868 23281 raft_consensus.cc:740] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:31.469476 23281 consensus_queue.cc:260] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.470500 23281 raft_consensus.cc:3060] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.473675 23354 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 5.
I20260709 12:12:31.474869 23394 leader_election.cc:304] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:31.474984 23281 raft_consensus.cc:2468] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 5.
I20260709 12:12:31.475665 23595 raft_consensus.cc:2804] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:31.477618 23595 raft_consensus.cc:697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:31.478461 23595 consensus_queue.cc:237] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.486553 23178 catalog_manager.cc:5697] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 4 to 5, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 5 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:31.628713 23615 raft_consensus.cc:993] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:31.629151 23615 raft_consensus.cc:1081] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Signalling peer b4d5867a2db442d291806a1ac0039588 to start an election
I20260709 12:12:31.630424 23354 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0ead34498294751a5df537c369db528"
dest_uuid: "b4d5867a2db442d291806a1ac0039588"
 from {username='slave'} at 127.0.0.1:54584
I20260709 12:12:31.630867 23354 raft_consensus.cc:493] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:31.631135 23354 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.637719 23354 raft_consensus.cc:515] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.639617 23354 leader_election.cc:290] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 4 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67:41953)
I20260709 12:12:31.641230 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:31.641422 23428 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0ead34498294751a5df537c369db528" candidate_uuid: "b4d5867a2db442d291806a1ac0039588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
I20260709 12:12:31.641815 23281 raft_consensus.cc:3055] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:31.642079 23428 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.642133 23281 raft_consensus.cc:740] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:31.642866 23281 consensus_queue.cc:260] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.643750 23281 raft_consensus.cc:3060] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:31.646943 23428 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 4.
I20260709 12:12:31.647940 23281 raft_consensus.cc:2468] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4d5867a2db442d291806a1ac0039588 in term 4.
I20260709 12:12:31.647959 23321 leader_election.cc:304] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:31.648694 23593 raft_consensus.cc:2804] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:31.649070 23593 raft_consensus.cc:697] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 4 LEADER]: Becoming Leader. State: Replica: b4d5867a2db442d291806a1ac0039588, State: Running, Role: LEADER
I20260709 12:12:31.649740 23593 consensus_queue.cc:237] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.656057 23178 catalog_manager.cc:5697] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 reported cstate change: term changed from 3 to 4, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to b4d5867a2db442d291806a1ac0039588 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "b4d5867a2db442d291806a1ac0039588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:31.715401 23622 raft_consensus.cc:993] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: : Instructing follower bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:31.715838 23622 raft_consensus.cc:1081] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Signalling peer bff7757456664d9a86c2c2b9bb12dcb3 to start an election
I20260709 12:12:31.717237 23428 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82"
dest_uuid: "bff7757456664d9a86c2c2b9bb12dcb3"
 from {username='slave'} at 127.0.0.1:51000
I20260709 12:12:31.717656 23428 raft_consensus.cc:493] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:31.717921 23428 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.722561 23428 raft_consensus.cc:515] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.724069 23428 leader_election.cc:290] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 5 election: Requested vote from peers 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559), b4d5867a2db442d291806a1ac0039588 (127.22.25.66:37367)
I20260709 12:12:31.725033 23281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "95f0c264b4dc4fe28d874331ff366741"
I20260709 12:12:31.725183 23354 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acc1b8b7fb534a65a834a1ad47b92d82" candidate_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b4d5867a2db442d291806a1ac0039588"
I20260709 12:12:31.725567 23281 raft_consensus.cc:3055] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:31.725641 23354 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.725834 23281 raft_consensus.cc:740] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 95f0c264b4dc4fe28d874331ff366741, State: Running, Role: LEADER
I20260709 12:12:31.726482 23281 consensus_queue.cc:260] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.727291 23281 raft_consensus.cc:3060] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:31.730093 23354 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 5.
I20260709 12:12:31.731103 23394 leader_election.cc:304] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4d5867a2db442d291806a1ac0039588, bff7757456664d9a86c2c2b9bb12dcb3; no voters: 
I20260709 12:12:31.731806 23281 raft_consensus.cc:2468] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff7757456664d9a86c2c2b9bb12dcb3 in term 5.
I20260709 12:12:31.732067 23595 raft_consensus.cc:2804] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:31.732553 23595 raft_consensus.cc:697] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 LEADER]: Becoming Leader. State: Replica: bff7757456664d9a86c2c2b9bb12dcb3, State: Running, Role: LEADER
I20260709 12:12:31.733239 23595 consensus_queue.cc:237] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } }
I20260709 12:12:31.739934 23178 catalog_manager.cc:5697] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 reported cstate change: term changed from 4 to 5, leader changed from 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65) to bff7757456664d9a86c2c2b9bb12dcb3 (127.22.25.67). New cstate: current_term: 5 leader_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:31.772838 23281 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:31.774039 23593 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:31.784157 23428 raft_consensus.cc:1275] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:31.786526 23623 consensus_queue.cc:1048] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:32.042901 23281 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:32.044400 23595 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:32.056612 23354 raft_consensus.cc:1275] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:32.057763 23639 consensus_queue.cc:1048] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:32.138104 23354 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:32.139424 23639 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d5867a2db442d291806a1ac0039588" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:32.142694 23281 raft_consensus.cc:1275] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Refusing update from remote peer bff7757456664d9a86c2c2b9bb12dcb3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:32.144381 23629 consensus_queue.cc:1048] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:32.241214 23430 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:32.242770 23643 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff7757456664d9a86c2c2b9bb12dcb3" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 41953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:32.252369 23281 raft_consensus.cc:1275] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Refusing update from remote peer b4d5867a2db442d291806a1ac0039588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:32.253959 23623 consensus_queue.cc:1048] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95f0c264b4dc4fe28d874331ff366741" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 36559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:32.355118 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:32.360394 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:32.360814 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:33.361999 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:33.368810 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:33.369306 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:34.370496 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:34.375888 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:34.376307 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:35.377138 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:35.382750 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:35.383214 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:36.384181 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:36.390038 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:36.390486 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:37.391396 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:37.397300 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:37.397722 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:38.398952 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:38.404814 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:38.405215 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:39.406188 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:39.411761 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:39.412142 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:40.413022 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:40.418658 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:40.419183 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:41.451871 22629 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:12:41.452488 22629 tablet_server.cc:179] TabletServer@127.22.25.65:0 shutting down...
I20260709 12:12:41.478694 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:41.479470 22629 tablet_replica.cc:333] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.480299 22629 raft_consensus.cc:2243] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.481303 22629 raft_consensus.cc:2272] T 4908b7f19700429584ac45090f98580d P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.483340 22629 tablet_replica.cc:333] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.484154 22629 raft_consensus.cc:2243] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.484686 22629 raft_consensus.cc:2272] T a697b070e4cf4d599a7959216dafa9b7 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.486958 22629 tablet_replica.cc:333] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.487618 22629 raft_consensus.cc:2243] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.488169 22629 raft_consensus.cc:2272] T acc1b8b7fb534a65a834a1ad47b92d82 P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.490406 22629 tablet_replica.cc:333] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.490964 22629 raft_consensus.cc:2243] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.491431 22629 raft_consensus.cc:2272] T a0ead34498294751a5df537c369db528 P 95f0c264b4dc4fe28d874331ff366741 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.493180 22629 tablet_replica.cc:333] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.493637 22629 raft_consensus.cc:2243] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.494423 22629 raft_consensus.cc:2272] T 48cde435c4ef4b40b4c2c179b305b37e P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.496493 22629 tablet_replica.cc:333] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.496970 22629 raft_consensus.cc:2243] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.497390 22629 raft_consensus.cc:2272] T bf956843dd6547a19fd10e4fd69dc0bb P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.499095 22629 tablet_replica.cc:333] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.499555 22629 raft_consensus.cc:2243] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.500362 22629 raft_consensus.cc:2272] T a60afbcca2b847fe93f7451e37f98b79 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.502068 22629 tablet_replica.cc:333] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.502530 22629 raft_consensus.cc:2243] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.502959 22629 raft_consensus.cc:2272] T c6a841b5028c46c0927d865e58b0f3c9 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.505155 22629 tablet_replica.cc:333] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.505596 22629 raft_consensus.cc:2243] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.506366 22629 raft_consensus.cc:2272] T 5ae90aa7d14a48a4ace7673c6f365c06 P 95f0c264b4dc4fe28d874331ff366741 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.508148 22629 tablet_replica.cc:333] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741: stopping tablet replica
I20260709 12:12:41.508543 22629 raft_consensus.cc:2243] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.508919 22629 raft_consensus.cc:2272] T 03866111a0a54ab2ad035e9c53d7c6ce P 95f0c264b4dc4fe28d874331ff366741 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.528041 22629 tablet_server.cc:196] TabletServer@127.22.25.65:0 shutdown complete.
I20260709 12:12:41.552927 22629 tablet_server.cc:179] TabletServer@127.22.25.66:0 shutting down...
I20260709 12:12:41.578171 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:41.578845 22629 tablet_replica.cc:333] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.579523 22629 raft_consensus.cc:2243] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.580399 22629 raft_consensus.cc:2272] T bf956843dd6547a19fd10e4fd69dc0bb P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.582630 22629 tablet_replica.cc:333] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.583189 22629 raft_consensus.cc:2243] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.583642 22629 raft_consensus.cc:2272] T 48cde435c4ef4b40b4c2c179b305b37e P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.586704 22629 tablet_replica.cc:333] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.587265 22629 raft_consensus.cc:2243] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.587776 22629 raft_consensus.cc:2272] T c6a841b5028c46c0927d865e58b0f3c9 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.589622 22629 tablet_replica.cc:333] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.590186 22629 raft_consensus.cc:2243] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.590624 22629 raft_consensus.cc:2272] T a60afbcca2b847fe93f7451e37f98b79 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.592684 22629 tablet_replica.cc:333] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.593207 22629 raft_consensus.cc:2243] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.593962 22629 raft_consensus.cc:2272] T a697b070e4cf4d599a7959216dafa9b7 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.595700 22629 tablet_replica.cc:333] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.596192 22629 raft_consensus.cc:2243] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.596650 22629 raft_consensus.cc:2272] T acc1b8b7fb534a65a834a1ad47b92d82 P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.598304 22629 tablet_replica.cc:333] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.598832 22629 raft_consensus.cc:2243] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.599531 22629 raft_consensus.cc:2272] T a0ead34498294751a5df537c369db528 P b4d5867a2db442d291806a1ac0039588 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.601892 22629 tablet_replica.cc:333] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.602460 22629 raft_consensus.cc:2243] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.603022 22629 raft_consensus.cc:2272] T 4908b7f19700429584ac45090f98580d P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.604983 22629 tablet_replica.cc:333] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.605474 22629 raft_consensus.cc:2243] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.605914 22629 raft_consensus.cc:2272] T 03866111a0a54ab2ad035e9c53d7c6ce P b4d5867a2db442d291806a1ac0039588 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.607635 22629 tablet_replica.cc:333] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588: stopping tablet replica
I20260709 12:12:41.608052 22629 raft_consensus.cc:2243] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.608441 22629 raft_consensus.cc:2272] T 5ae90aa7d14a48a4ace7673c6f365c06 P b4d5867a2db442d291806a1ac0039588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.637633 22629 tablet_server.cc:196] TabletServer@127.22.25.66:0 shutdown complete.
I20260709 12:12:41.659236 22629 tablet_server.cc:179] TabletServer@127.22.25.67:0 shutting down...
I20260709 12:12:41.685119 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:41.685779 22629 tablet_replica.cc:333] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.686466 22629 raft_consensus.cc:2243] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.687345 22629 raft_consensus.cc:2272] T c6a841b5028c46c0927d865e58b0f3c9 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.689200 22629 tablet_replica.cc:333] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.689715 22629 raft_consensus.cc:2243] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.690156 22629 raft_consensus.cc:2272] T a60afbcca2b847fe93f7451e37f98b79 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.692011 22629 tablet_replica.cc:333] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.692548 22629 raft_consensus.cc:2243] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.692997 22629 raft_consensus.cc:2272] T a0ead34498294751a5df537c369db528 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.694876 22629 tablet_replica.cc:333] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.695410 22629 raft_consensus.cc:2243] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.695864 22629 raft_consensus.cc:2272] T 4908b7f19700429584ac45090f98580d P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.697602 22629 tablet_replica.cc:333] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.698139 22629 raft_consensus.cc:2243] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.698591 22629 raft_consensus.cc:2272] T 48cde435c4ef4b40b4c2c179b305b37e P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.700848 22629 tablet_replica.cc:333] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.701583 22629 raft_consensus.cc:2243] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.702288 22629 raft_consensus.cc:2272] T bf956843dd6547a19fd10e4fd69dc0bb P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:41.703105 23394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.25.65:36559: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260709 12:12:41.704829 22629 tablet_replica.cc:333] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.705425 22629 raft_consensus.cc:2243] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.705972 22629 raft_consensus.cc:2272] T a697b070e4cf4d599a7959216dafa9b7 P bff7757456664d9a86c2c2b9bb12dcb3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:41.706874 23394 consensus_peers.cc:597] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 -> Peer 95f0c264b4dc4fe28d874331ff366741 (127.22.25.65:36559): Couldn't send request to peer 95f0c264b4dc4fe28d874331ff366741. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:36559: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:41.707947 22629 tablet_replica.cc:333] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.708470 22629 raft_consensus.cc:2243] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.709256 22629 raft_consensus.cc:2272] T acc1b8b7fb534a65a834a1ad47b92d82 P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.711102 22629 tablet_replica.cc:333] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.711580 22629 raft_consensus.cc:2243] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.712529 22629 raft_consensus.cc:2272] T 03866111a0a54ab2ad035e9c53d7c6ce P bff7757456664d9a86c2c2b9bb12dcb3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.714385 22629 tablet_replica.cc:333] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3: stopping tablet replica
I20260709 12:12:41.714839 22629 raft_consensus.cc:2243] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:41.715288 22629 raft_consensus.cc:2272] T 5ae90aa7d14a48a4ace7673c6f365c06 P bff7757456664d9a86c2c2b9bb12dcb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.744971 22629 tablet_server.cc:196] TabletServer@127.22.25.67:0 shutdown complete.
I20260709 12:12:41.764688 22629 master.cc:562] Master@127.22.25.126:44009 shutting down...
I20260709 12:12:41.788092 22629 raft_consensus.cc:2243] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:12:41.788682 22629 raft_consensus.cc:2272] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:41.789033 22629 tablet_replica.cc:333] T 00000000000000000000000000000000 P abd11f7ba0bc48908eeb3acbdea7d9b9: stopping tablet replica
I20260709 12:12:41.808995 22629 master.cc:584] Master@127.22.25.126:44009 shutdown complete.
95f0c264b4dc4fe28d874331ff366741  5
b4d5867a2db442d291806a1ac0039588  4
bff7757456664d9a86c2c2b9bb12dcb3  1
95f0c264b4dc4fe28d874331ff366741  3
b4d5867a2db442d291806a1ac0039588  4
bff7757456664d9a86c2c2b9bb12dcb3  3
95f0c264b4dc4fe28d874331ff366741  8
b4d5867a2db442d291806a1ac0039588  1
bff7757456664d9a86c2c2b9bb12dcb3  1
95f0c264b4dc4fe28d874331ff366741  4
b4d5867a2db442d291806a1ac0039588  3
bff7757456664d9a86c2c2b9bb12dcb3  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28769 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:12:41.849666 22629 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.22.25.126:39113
I20260709 12:12:41.850987 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:41.856514 23659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:41.857575 23658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:41.858109 23661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:41.858525 22629 server_base.cc:1061] running on GCE node
I20260709 12:12:41.859942 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:41.860153 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:41.860296 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599161860281 us; error 0 us; skew 500 ppm
I20260709 12:12:41.860800 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:41.863837 22629 webserver.cc:533] Webserver started at http://127.22.25.126:35169/ using document root <none> and password file <none>
I20260709 12:12:41.864338 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:41.864521 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:41.864794 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:41.865989 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/master-0-root/instance:
uuid: "8f6194fc84354daa9cd0b57de46d8e32"
format_stamp: "Formatted at 2026-07-09 12:12:41 on dist-test-slave-sc22"
I20260709 12:12:41.870978 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20260709 12:12:41.874552 23666 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:41.875324 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:12:41.875622 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/master-0-root
uuid: "8f6194fc84354daa9cd0b57de46d8e32"
format_stamp: "Formatted at 2026-07-09 12:12:41 on dist-test-slave-sc22"
I20260709 12:12:41.875909 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:41.890420 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:41.891726 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:41.932528 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.126:39113
I20260709 12:12:41.932626 23717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.126:39113 every 8 connection(s)
I20260709 12:12:41.936373 23718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:41.946384 23718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Bootstrap starting.
I20260709 12:12:41.951115 23718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:41.955443 23718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: No bootstrap required, opened a new log
I20260709 12:12:41.957674 23718 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER }
I20260709 12:12:41.958115 23718 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:41.958343 23718 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6194fc84354daa9cd0b57de46d8e32, State: Initialized, Role: FOLLOWER
I20260709 12:12:41.959076 23718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER }
I20260709 12:12:41.959547 23718 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:12:41.959784 23718 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:12:41.960052 23718 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:41.965188 23718 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER }
I20260709 12:12:41.965776 23718 leader_election.cc:304] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6194fc84354daa9cd0b57de46d8e32; no voters: 
I20260709 12:12:41.966889 23718 leader_election.cc:290] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:12:41.967269 23721 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:41.968538 23721 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [term 1 LEADER]: Becoming Leader. State: Replica: 8f6194fc84354daa9cd0b57de46d8e32, State: Running, Role: LEADER
I20260709 12:12:41.969174 23721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER }
I20260709 12:12:41.969900 23718 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:12:41.973979 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6194fc84354daa9cd0b57de46d8e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER } }
I20260709 12:12:41.974104 23723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6194fc84354daa9cd0b57de46d8e32. Latest consensus state: current_term: 1 leader_uuid: "8f6194fc84354daa9cd0b57de46d8e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6194fc84354daa9cd0b57de46d8e32" member_type: VOTER } }
I20260709 12:12:41.974828 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:41.974857 23723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:41.977026 23728 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:12:41.982048 23728 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:12:41.986732 22629 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:12:41.991328 23728 catalog_manager.cc:1383] Generated new cluster ID: 0aea995f7dfd4ae5ada28fe16dd70d4d
I20260709 12:12:41.991684 23728 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:12:42.022110 23728 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:12:42.023401 23728 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:12:42.037365 23728 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Generated new TSK 0
I20260709 12:12:42.037969 23728 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:12:42.053234 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:42.059317 23739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:42.060482 23740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:42.061728 23742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:42.062743 22629 server_base.cc:1061] running on GCE node
I20260709 12:12:42.063731 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:42.063916 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:42.064072 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599162064054 us; error 0 us; skew 500 ppm
I20260709 12:12:42.064612 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:42.066931 22629 webserver.cc:533] Webserver started at http://127.22.25.65:42937/ using document root <none> and password file <none>
I20260709 12:12:42.067440 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:42.067703 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:42.068034 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:42.069247 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/instance:
uuid: "0ee4ab0747914e7abc642312517b5e25"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.074168 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:12:42.078282 23747 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.079100 22629 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:12:42.079393 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root
uuid: "0ee4ab0747914e7abc642312517b5e25"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.079707 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:42.098805 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:42.100180 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:42.101810 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:42.104215 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:42.104423 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.104655 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:42.104825 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.149694 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.65:35817
I20260709 12:12:42.149804 23809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.65:35817 every 8 connection(s)
I20260709 12:12:42.154978 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:42.162307 23814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:42.164074 23815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:42.166841 22629 server_base.cc:1061] running on GCE node
W20260709 12:12:42.168354 23817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:42.168571 23810 heartbeater.cc:344] Connected to a master server at 127.22.25.126:39113
I20260709 12:12:42.168998 23810 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:42.169317 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:42.169543 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:42.169756 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599162169736 us; error 0 us; skew 500 ppm
I20260709 12:12:42.169792 23810 heartbeater.cc:507] Master 127.22.25.126:39113 requested a full tablet report, sending...
I20260709 12:12:42.170430 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:42.172225 23683 ts_manager.cc:194] Registered new tserver with Master: 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:42.173501 22629 webserver.cc:533] Webserver started at http://127.22.25.66:37567/ using document root <none> and password file <none>
I20260709 12:12:42.174150 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:42.174391 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:42.174686 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:42.174741 23683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58540
I20260709 12:12:42.176263 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/instance:
uuid: "438159d8de3c4027bdcf57eb81dedf86"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.181443 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:12:42.185070 23822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.185896 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:12:42.186195 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root
uuid: "438159d8de3c4027bdcf57eb81dedf86"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.186488 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:42.202489 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:42.203776 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:42.205288 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:42.207623 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:42.207839 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.208106 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:42.208261 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.250375 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.66:44897
I20260709 12:12:42.250442 23884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.66:44897 every 8 connection(s)
I20260709 12:12:42.255448 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:42.262809 23888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:42.264658 23889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:42.265966 22629 server_base.cc:1061] running on GCE node
W20260709 12:12:42.266469 23891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:42.267510 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:42.267767 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:42.267935 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599162267917 us; error 0 us; skew 500 ppm
I20260709 12:12:42.268540 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:42.268587 23885 heartbeater.cc:344] Connected to a master server at 127.22.25.126:39113
I20260709 12:12:42.268996 23885 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:42.269738 23885 heartbeater.cc:507] Master 127.22.25.126:39113 requested a full tablet report, sending...
I20260709 12:12:42.271166 22629 webserver.cc:533] Webserver started at http://127.22.25.67:32921/ using document root <none> and password file <none>
I20260709 12:12:42.271739 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:42.271932 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:42.271946 23683 ts_manager.cc:194] Registered new tserver with Master: 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:42.272244 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:42.273393 23683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58556
I20260709 12:12:42.273447 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/instance:
uuid: "9e0bf56d58c5480a8d088c85566558da"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.278113 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:12:42.281442 23896 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.282361 22629 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:12:42.282648 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root
uuid: "9e0bf56d58c5480a8d088c85566558da"
format_stamp: "Formatted at 2026-07-09 12:12:42 on dist-test-slave-sc22"
I20260709 12:12:42.282909 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:42.300313 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:42.301534 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:42.302980 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:42.305267 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:42.305465 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.305687 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:42.305857 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:42.346515 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.67:40463
I20260709 12:12:42.346619 23958 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.67:40463 every 8 connection(s)
I20260709 12:12:42.360306 23959 heartbeater.cc:344] Connected to a master server at 127.22.25.126:39113
I20260709 12:12:42.360671 23959 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:42.361865 23959 heartbeater.cc:507] Master 127.22.25.126:39113 requested a full tablet report, sending...
I20260709 12:12:42.364017 23683 ts_manager.cc:194] Registered new tserver with Master: 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:42.364691 22629 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015194167s
I20260709 12:12:42.365417 23683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58558
I20260709 12:12:42.391501 23683 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58568:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:12:42.397495 23683 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:12:42.645380 23849 tablet_service.cc:1511] Processing CreateTablet for tablet 643746a7a11243df94746cf1081fc078 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:42.648206 23843 tablet_service.cc:1511] Processing CreateTablet for tablet 9d72845b5ad4457bb71a7ad5f5152b4c (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:42.648885 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 98e5bc86804f4f979f58779caa696814 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:42.649789 23842 tablet_service.cc:1511] Processing CreateTablet for tablet 34d6300a13a3401283a46bf3d2a07e82 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:42.650645 23844 tablet_service.cc:1511] Processing CreateTablet for tablet 71cb2ce294574daa8d7244ae252e8aa6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:42.651286 23841 tablet_service.cc:1511] Processing CreateTablet for tablet 8368f2e695204683affed6a9bd84ba96 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:42.652361 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8a875cf4b4acea62c1086994499e4 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:42.652812 23924 tablet_service.cc:1511] Processing CreateTablet for tablet b54dd3a3d61243bea2dae318ccea3958 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:42.653825 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643746a7a11243df94746cf1081fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.653896 23846 tablet_service.cc:1511] Processing CreateTablet for tablet de3c70defd864769b26b52b7f36548cd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:42.655453 23847 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ef29919aa42aaa2b7545c10e39eff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:42.654374 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54dd3a3d61243bea2dae318ccea3958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.645131 23850 tablet_service.cc:1511] Processing CreateTablet for tablet b54dd3a3d61243bea2dae318ccea3958 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:42.661036 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 643746a7a11243df94746cf1081fc078 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:42.662498 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643746a7a11243df94746cf1081fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.664623 23774 tablet_service.cc:1511] Processing CreateTablet for tablet 643746a7a11243df94746cf1081fc078 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:42.666231 23774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643746a7a11243df94746cf1081fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.667089 23775 tablet_service.cc:1511] Processing CreateTablet for tablet b54dd3a3d61243bea2dae318ccea3958 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:42.668645 23775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54dd3a3d61243bea2dae318ccea3958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.668834 23772 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ef29919aa42aaa2b7545c10e39eff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:42.670545 23772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ef29919aa42aaa2b7545c10e39eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.670806 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 98e5bc86804f4f979f58779caa696814 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:42.672243 23770 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8a875cf4b4acea62c1086994499e4 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:42.673028 23920 tablet_service.cc:1511] Processing CreateTablet for tablet de3c70defd864769b26b52b7f36548cd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:42.673970 23771 tablet_service.cc:1511] Processing CreateTablet for tablet de3c70defd864769b26b52b7f36548cd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:42.674470 23920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3c70defd864769b26b52b7f36548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.675323 23771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3c70defd864769b26b52b7f36548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.677489 23919 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8a875cf4b4acea62c1086994499e4 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:42.679330 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8a875cf4b4acea62c1086994499e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.679877 23921 tablet_service.cc:1511] Processing CreateTablet for tablet cc9ef29919aa42aaa2b7545c10e39eff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:42.655371 23846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3c70defd864769b26b52b7f36548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.668880 23922 tablet_service.cc:1511] Processing CreateTablet for tablet 98e5bc86804f4f979f58779caa696814 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:42.687490 23769 tablet_service.cc:1511] Processing CreateTablet for tablet 71cb2ce294574daa8d7244ae252e8aa6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:42.688735 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8a875cf4b4acea62c1086994499e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.689448 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98e5bc86804f4f979f58779caa696814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.689996 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ef29919aa42aaa2b7545c10e39eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.691915 23769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cb2ce294574daa8d7244ae252e8aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.687606 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 71cb2ce294574daa8d7244ae252e8aa6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:42.695906 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cb2ce294574daa8d7244ae252e8aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.700510 23922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98e5bc86804f4f979f58779caa696814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.703794 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98e5bc86804f4f979f58779caa696814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.704795 23841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8368f2e695204683affed6a9bd84ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.706965 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cb2ce294574daa8d7244ae252e8aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.707757 23915 tablet_service.cc:1511] Processing CreateTablet for tablet 8368f2e695204683affed6a9bd84ba96 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:42.709200 23915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8368f2e695204683affed6a9bd84ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.708847 23916 tablet_service.cc:1511] Processing CreateTablet for tablet 34d6300a13a3401283a46bf3d2a07e82 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:42.710840 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b54dd3a3d61243bea2dae318ccea3958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.710997 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d6300a13a3401283a46bf3d2a07e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.707199 23917 tablet_service.cc:1511] Processing CreateTablet for tablet 9d72845b5ad4457bb71a7ad5f5152b4c (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:42.712718 23917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d72845b5ad4457bb71a7ad5f5152b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.719996 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8a875cf4b4acea62c1086994499e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.722069 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d6300a13a3401283a46bf3d2a07e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.734258 23768 tablet_service.cc:1511] Processing CreateTablet for tablet 9d72845b5ad4457bb71a7ad5f5152b4c (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:42.737578 23767 tablet_service.cc:1511] Processing CreateTablet for tablet 34d6300a13a3401283a46bf3d2a07e82 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:42.738693 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ef29919aa42aaa2b7545c10e39eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.743762 23766 tablet_service.cc:1511] Processing CreateTablet for tablet 8368f2e695204683affed6a9bd84ba96 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:42.744045 23768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d72845b5ad4457bb71a7ad5f5152b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.747459 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d6300a13a3401283a46bf3d2a07e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.748183 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8368f2e695204683affed6a9bd84ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.753280 23843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d72845b5ad4457bb71a7ad5f5152b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.854434 23769 tablet_service.cc:1511] Processing CreateTablet for tablet c395cd7fe30b41869bce6dac359f2cf8 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:42.861300 23771 tablet_service.cc:1511] Processing CreateTablet for tablet 233089f4b82a4b1996ad9129dc392819 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:42.864494 23772 tablet_service.cc:1511] Processing CreateTablet for tablet 861b3b9e09c14ef99794d4c8d50d59ad (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:42.874277 23769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c395cd7fe30b41869bce6dac359f2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.877262 23842 tablet_service.cc:1511] Processing CreateTablet for tablet c395cd7fe30b41869bce6dac359f2cf8 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:42.878670 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c395cd7fe30b41869bce6dac359f2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.882076 23915 tablet_service.cc:1511] Processing CreateTablet for tablet c395cd7fe30b41869bce6dac359f2cf8 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:42.888511 23915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c395cd7fe30b41869bce6dac359f2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.892199 23772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861b3b9e09c14ef99794d4c8d50d59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.893594 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 1b93ab9c24064567b717af205bc2ed2a (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:42.894976 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b93ab9c24064567b717af205bc2ed2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.900305 23844 tablet_service.cc:1511] Processing CreateTablet for tablet 233089f4b82a4b1996ad9129dc392819 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:42.901700 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233089f4b82a4b1996ad9129dc392819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.905717 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 233089f4b82a4b1996ad9129dc392819 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:42.907151 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233089f4b82a4b1996ad9129dc392819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.908047 23977 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:42.910693 23917 tablet_service.cc:1511] Processing CreateTablet for tablet 861b3b9e09c14ef99794d4c8d50d59ad (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:42.913338 23922 tablet_service.cc:1511] Processing CreateTablet for tablet 1567865ea02a4884996ae9d8beb9fa29 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:42.913338 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 1b93ab9c24064567b717af205bc2ed2a (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:42.918740 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b93ab9c24064567b717af205bc2ed2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.919096 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 1b93ab9c24064567b717af205bc2ed2a (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:42.919128 23841 tablet_service.cc:1511] Processing CreateTablet for tablet e1031da3c683416ebfc283d98b45867d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:42.921108 23917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861b3b9e09c14ef99794d4c8d50d59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.915699 23921 tablet_service.cc:1511] Processing CreateTablet for tablet e1031da3c683416ebfc283d98b45867d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:42.928887 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1031da3c683416ebfc283d98b45867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.929472 23850 tablet_service.cc:1511] Processing CreateTablet for tablet 1567865ea02a4884996ae9d8beb9fa29 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:42.930922 23922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1567865ea02a4884996ae9d8beb9fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.929505 23849 tablet_service.cc:1511] Processing CreateTablet for tablet c6f2d1f3ed104c1e99f82ebe04843248 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:42.934831 23976 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:42.944348 23975 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:42.946030 23770 tablet_service.cc:1511] Processing CreateTablet for tablet e1031da3c683416ebfc283d98b45867d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:42.947374 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1031da3c683416ebfc283d98b45867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.954632 23924 tablet_service.cc:1511] Processing CreateTablet for tablet 0b23f5ca4f084b8e96cc4a3099e99074 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:42.949283 23916 tablet_service.cc:1511] Processing CreateTablet for tablet c6f2d1f3ed104c1e99f82ebe04843248 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:42.950448 23767 tablet_service.cc:1511] Processing CreateTablet for tablet 1567865ea02a4884996ae9d8beb9fa29 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:42.956670 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1567865ea02a4884996ae9d8beb9fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.958699 23841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1031da3c683416ebfc283d98b45867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.960464 23774 tablet_service.cc:1511] Processing CreateTablet for tablet c6f2d1f3ed104c1e99f82ebe04843248 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:42.960767 23920 tablet_service.cc:1511] Processing CreateTablet for tablet 78f1c4dc751a4f4a93a446a785edda51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:42.915812 23843 tablet_service.cc:1511] Processing CreateTablet for tablet 861b3b9e09c14ef99794d4c8d50d59ad (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:42.962406 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 0b23f5ca4f084b8e96cc4a3099e99074 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:42.965103 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f2d1f3ed104c1e99f82ebe04843248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.996394 23919 tablet_service.cc:1511] Processing CreateTablet for tablet 8f333e0a484248ce88f0905e18500838 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:42.997149 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1567865ea02a4884996ae9d8beb9fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.998812 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b93ab9c24064567b717af205bc2ed2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.975481 23846 tablet_service.cc:1511] Processing CreateTablet for tablet 0b23f5ca4f084b8e96cc4a3099e99074 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:43.002861 23846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b23f5ca4f084b8e96cc4a3099e99074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.998034 23917 tablet_service.cc:1511] Processing CreateTablet for tablet c7203e0ca92840388d931c03974d32ff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:42.962342 23771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233089f4b82a4b1996ad9129dc392819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.006498 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b23f5ca4f084b8e96cc4a3099e99074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.007114 23774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f2d1f3ed104c1e99f82ebe04843248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.006850 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 78f1c4dc751a4f4a93a446a785edda51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:43.008085 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f1c4dc751a4f4a93a446a785edda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.975481 23768 tablet_service.cc:1511] Processing CreateTablet for tablet 78f1c4dc751a4f4a93a446a785edda51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:43.009554 23768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f1c4dc751a4f4a93a446a785edda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.978621 23977 tablet_bootstrap.cc:654] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:42.982203 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:43.015926 23775 tablet_service.cc:1511] Processing CreateTablet for tablet 8f333e0a484248ce88f0905e18500838 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:43.017262 23775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f333e0a484248ce88f0905e18500838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.992843 23976 tablet_bootstrap.cc:654] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:42.998845 23975 tablet_bootstrap.cc:654] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.021082 23843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861b3b9e09c14ef99794d4c8d50d59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:42.993548 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b23f5ca4f084b8e96cc4a3099e99074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.021921 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f2d1f3ed104c1e99f82ebe04843248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.022522 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f333e0a484248ce88f0905e18500838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.023325 23917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7203e0ca92840388d931c03974d32ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.026841 23920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f1c4dc751a4f4a93a446a785edda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.035490 23841 tablet_service.cc:1511] Processing CreateTablet for tablet 8f333e0a484248ce88f0905e18500838 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:43.036952 23841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f333e0a484248ce88f0905e18500838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.037943 23772 tablet_service.cc:1511] Processing CreateTablet for tablet c7203e0ca92840388d931c03974d32ff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:43.039213 23772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7203e0ca92840388d931c03974d32ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.048810 23977 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:43.049451 23977 ts_tablet_manager.cc:1403] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.190s	user 0.007s	sys 0.006s
I20260709 12:12:43.050388 23975 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.050860 23975 ts_tablet_manager.cc:1403] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.107s	user 0.002s	sys 0.008s
I20260709 12:12:43.051888 23915 tablet_service.cc:1511] Processing CreateTablet for tablet 526cc198da134bb590358ed73c23d59e (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:43.053262 23915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526cc198da134bb590358ed73c23d59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.056716 23770 tablet_service.cc:1511] Processing CreateTablet for tablet 20fc63afa03b45c0bf3f58aae82f3e3d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:43.058051 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fc63afa03b45c0bf3f58aae82f3e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.059679 23976 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:43.060230 23976 ts_tablet_manager.cc:1403] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.126s	user 0.004s	sys 0.008s
I20260709 12:12:43.040577 23766 tablet_service.cc:1511] Processing CreateTablet for tablet 526cc198da134bb590358ed73c23d59e (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:43.064677 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526cc198da134bb590358ed73c23d59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.042783 23842 tablet_service.cc:1511] Processing CreateTablet for tablet 526cc198da134bb590358ed73c23d59e (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:43.069831 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526cc198da134bb590358ed73c23d59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.069454 23847 tablet_service.cc:1511] Processing CreateTablet for tablet c7203e0ca92840388d931c03974d32ff (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:43.070873 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7203e0ca92840388d931c03974d32ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.071935 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 20fc63afa03b45c0bf3f58aae82f3e3d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:43.073289 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fc63afa03b45c0bf3f58aae82f3e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.053045 23977 raft_consensus.cc:359] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.076992 23977 raft_consensus.cc:385] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.077419 23977 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.077275 23767 tablet_service.cc:1511] Processing CreateTablet for tablet fc09548ba8eb4f01bf6d74a33d072dda (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:43.078821 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc09548ba8eb4f01bf6d74a33d072dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.078650 23977 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.094913 23975 raft_consensus.cc:359] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.095743 23975 raft_consensus.cc:385] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.095556 23849 tablet_service.cc:1511] Processing CreateTablet for tablet 20fc63afa03b45c0bf3f58aae82f3e3d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:43.096045 23975 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.096926 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fc63afa03b45c0bf3f58aae82f3e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.097071 23975 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.105515 23921 tablet_service.cc:1511] Processing CreateTablet for tablet fc09548ba8eb4f01bf6d74a33d072dda (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:43.063391 23976 raft_consensus.cc:359] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.108286 23976 raft_consensus.cc:385] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.108657 23976 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.109674 23976 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.136845 23975 ts_tablet_manager.cc:1434] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.086s	user 0.003s	sys 0.007s
I20260709 12:12:43.138458 23850 tablet_service.cc:1511] Processing CreateTablet for tablet c2844521776744308787188de9041ebd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:43.112473 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc09548ba8eb4f01bf6d74a33d072dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.141209 23982 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.141796 23982 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.117413 23844 tablet_service.cc:1511] Processing CreateTablet for tablet fc09548ba8eb4f01bf6d74a33d072dda (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:43.121218 23769 tablet_service.cc:1511] Processing CreateTablet for tablet 20edc48b2f884789ac3daead4b5c3102 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:43.123294 23810 heartbeater.cc:499] Master 127.22.25.126:39113 was elected leader, sending a full tablet report...
I20260709 12:12:43.124867 23885 heartbeater.cc:499] Master 127.22.25.126:39113 was elected leader, sending a full tablet report...
I20260709 12:12:43.154354 23769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20edc48b2f884789ac3daead4b5c3102. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:43.155874 23811 tablet.cc:2406] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:12:43.124634 23773 tablet_service.cc:1511] Processing CreateTablet for tablet c2844521776744308787188de9041ebd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:43.160939 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2844521776744308787188de9041ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.126557 23977 ts_tablet_manager.cc:1434] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.000s
I20260709 12:12:43.128615 23918 tablet_service.cc:1511] Processing CreateTablet for tablet c2844521776744308787188de9041ebd (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:43.133776 23916 tablet_service.cc:1511] Processing CreateTablet for tablet 20edc48b2f884789ac3daead4b5c3102 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:43.169093 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2844521776744308787188de9041ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.172263 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20edc48b2f884789ac3daead4b5c3102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.138406 23975 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.182577 23981 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.183109 23981 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.208523 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2844521776744308787188de9041ebd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.209401 23977 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:43.185266 23982 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.218588 23981 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.237000 23975 tablet_bootstrap.cc:654] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.203538 23848 tablet_service.cc:1511] Processing CreateTablet for tablet c4369a8cdaaa45beac1359775fa54781 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:43.245848 23976 ts_tablet_manager.cc:1434] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.185s	user 0.017s	sys 0.029s
I20260709 12:12:43.246848 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc09548ba8eb4f01bf6d74a33d072dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.247373 23959 heartbeater.cc:499] Master 127.22.25.126:39113 was elected leader, sending a full tablet report...
I20260709 12:12:43.252311 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 20edc48b2f884789ac3daead4b5c3102 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:43.257073 23924 tablet_service.cc:1511] Processing CreateTablet for tablet c4369a8cdaaa45beac1359775fa54781 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:43.258564 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4369a8cdaaa45beac1359775fa54781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.267403 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0370e122479081184511cb1224f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:43.267720 23770 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0370e122479081184511cb1224f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:43.268905 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0370e122479081184511cb1224f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.269045 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0370e122479081184511cb1224f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.276440 23767 tablet_service.cc:1511] Processing CreateTablet for tablet c4369a8cdaaa45beac1359775fa54781 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:43.282816 23768 tablet_service.cc:1511] Processing CreateTablet for tablet 77aa2cbe928a45d4b1652ca10a80aa99 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:43.283528 23849 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0370e122479081184511cb1224f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:43.285626 23977 tablet_bootstrap.cc:654] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.287842 23976 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:43.293926 23768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77aa2cbe928a45d4b1652ca10a80aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.310770 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:43.289460 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20edc48b2f884789ac3daead4b5c3102. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:43.312892 23749 leader_election.cc:343] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.312938 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4369a8cdaaa45beac1359775fa54781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.316181 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0370e122479081184511cb1224f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.319335 23847 tablet_service.cc:1511] Processing CreateTablet for tablet 77aa2cbe928a45d4b1652ca10a80aa99 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:43.321015 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77aa2cbe928a45d4b1652ca10a80aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.326303 23919 tablet_service.cc:1511] Processing CreateTablet for tablet 77aa2cbe928a45d4b1652ca10a80aa99 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:43.327836 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77aa2cbe928a45d4b1652ca10a80aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.313295 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4369a8cdaaa45beac1359775fa54781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.339395 23766 tablet_service.cc:1511] Processing CreateTablet for tablet f5bb29269e8a4aa1b79f91e4ed687053 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:43.345746 23842 tablet_service.cc:1511] Processing CreateTablet for tablet f5bb29269e8a4aa1b79f91e4ed687053 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:43.346513 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:43.347141 23771 tablet_service.cc:1511] Processing CreateTablet for tablet c8b83386d2fa4ed2a212fbe8873239ea (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:43.347088 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bb29269e8a4aa1b79f91e4ed687053. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:43.348780 23824 leader_election.cc:343] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.350602 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:43.352605 23751 leader_election.cc:343] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.353116 23751 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:43.353952 23982 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:43.354398 23976 tablet_bootstrap.cc:654] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.366667 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.366801 23924 tablet_service.cc:1511] Processing CreateTablet for tablet f5bb29269e8a4aa1b79f91e4ed687053 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:43.367734 23771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b83386d2fa4ed2a212fbe8873239ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.368184 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bb29269e8a4aa1b79f91e4ed687053. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:43.368777 23825 leader_election.cc:343] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.369242 23825 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:43.370513 23981 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:43.371757 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5bb29269e8a4aa1b79f91e4ed687053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.371969 23977 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:43.372568 23977 ts_tablet_manager.cc:1403] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.179s	user 0.011s	sys 0.021s
I20260709 12:12:43.375728 23977 raft_consensus.cc:359] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.376471 23977 raft_consensus.cc:385] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.376750 23977 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.387140 23977 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.397003 23977 ts_tablet_manager.cc:1434] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.005s
I20260709 12:12:43.398176 23977 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:43.403949 23975 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.404502 23975 ts_tablet_manager.cc:1403] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.266s	user 0.013s	sys 0.005s
I20260709 12:12:43.407433 23975 raft_consensus.cc:359] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.408201 23975 raft_consensus.cc:385] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.408506 23975 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.412241 23847 tablet_service.cc:1511] Processing CreateTablet for tablet c8b83386d2fa4ed2a212fbe8873239ea (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:43.416431 23975 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.418182 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b83386d2fa4ed2a212fbe8873239ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.418309 23918 tablet_service.cc:1511] Processing CreateTablet for tablet c8b83386d2fa4ed2a212fbe8873239ea (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:43.419098 23975 ts_tablet_manager.cc:1434] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:12:43.420253 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b83386d2fa4ed2a212fbe8873239ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.430943 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 3aacc8d80dd2433b94e6d07927304b47 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:43.432420 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aacc8d80dd2433b94e6d07927304b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.435671 23924 tablet_service.cc:1511] Processing CreateTablet for tablet 3aacc8d80dd2433b94e6d07927304b47 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:43.437080 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aacc8d80dd2433b94e6d07927304b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.440698 23975 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.443158 23976 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:43.443789 23976 ts_tablet_manager.cc:1403] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.170s	user 0.010s	sys 0.009s
I20260709 12:12:43.446730 23976 raft_consensus.cc:359] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.463187 23976 raft_consensus.cc:385] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.463650 23976 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.464560 23976 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.466831 23976 ts_tablet_manager.cc:1434] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:12:43.467734 23976 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:43.454454 23850 tablet_service.cc:1511] Processing CreateTablet for tablet d8341d6ef4764792b1bd6ff196e8af38 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:43.454911 23771 tablet_service.cc:1511] Processing CreateTablet for tablet 3aacc8d80dd2433b94e6d07927304b47 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:43.456151 23919 tablet_service.cc:1511] Processing CreateTablet for tablet d8341d6ef4764792b1bd6ff196e8af38 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:43.457544 23975 tablet_bootstrap.cc:654] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.450639 23770 tablet_service.cc:1511] Processing CreateTablet for tablet d8341d6ef4764792b1bd6ff196e8af38 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:43.452730 23977 tablet_bootstrap.cc:654] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.469136 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8341d6ef4764792b1bd6ff196e8af38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.469187 23771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aacc8d80dd2433b94e6d07927304b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.476325 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8341d6ef4764792b1bd6ff196e8af38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.477828 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8341d6ef4764792b1bd6ff196e8af38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.478224 23766 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed3fc49956b40aa86095410b7c7e5fa (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:43.485638 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed3fc49956b40aa86095410b7c7e5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.486207 23977 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:43.486809 23977 ts_tablet_manager.cc:1403] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.008s
I20260709 12:12:43.489655 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed3fc49956b40aa86095410b7c7e5fa (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:43.481359 23976 tablet_bootstrap.cc:654] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.496389 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed3fc49956b40aa86095410b7c7e5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.489846 23977 raft_consensus.cc:359] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.498894 23977 raft_consensus.cc:385] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.499275 23977 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.492228 23983 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.507921 23983 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.510739 23983 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.480841 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed3fc49956b40aa86095410b7c7e5fa (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:43.500733 23975 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.491466 23767 tablet_service.cc:1511] Processing CreateTablet for tablet 4e60a1c6520545729d27474d8f237ce5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:43.502890 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 4e60a1c6520545729d27474d8f237ce5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:43.504161 23916 tablet_service.cc:1511] Processing CreateTablet for tablet 4e60a1c6520545729d27474d8f237ce5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:43.512624 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed3fc49956b40aa86095410b7c7e5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.512756 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e60a1c6520545729d27474d8f237ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.513669 23977 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.515566 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e60a1c6520545729d27474d8f237ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.516252 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e60a1c6520545729d27474d8f237ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.518837 23977 ts_tablet_manager.cc:1434] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 12:12:43.527079 23977 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:43.512003 23975 ts_tablet_manager.cc:1403] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.072s	user 0.011s	sys 0.001s
I20260709 12:12:43.533788 23975 raft_consensus.cc:359] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.534583 23975 raft_consensus.cc:385] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.534888 23975 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.542925 23976 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:43.543431 23976 ts_tablet_manager.cc:1403] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.076s	user 0.006s	sys 0.006s
I20260709 12:12:43.546506 23976 raft_consensus.cc:359] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.547205 23976 raft_consensus.cc:385] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.549759 23976 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.550514 23976 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.551106 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 3eeb67716b0e4f49aa1a4bd5815f684b (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:43.552469 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eeb67716b0e4f49aa1a4bd5815f684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.552840 23976 ts_tablet_manager.cc:1434] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:12:43.553817 23976 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:43.556654 23847 tablet_service.cc:1511] Processing CreateTablet for tablet d6d3d4f634284532bbf6ad7d4ac19214 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:43.558014 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6d3d4f634284532bbf6ad7d4ac19214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.565444 23977 tablet_bootstrap.cc:654] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.572108 23770 tablet_service.cc:1511] Processing CreateTablet for tablet d6d3d4f634284532bbf6ad7d4ac19214 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:43.573190 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6d3d4f634284532bbf6ad7d4ac19214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.541049 23975 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.579981 23982 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.580487 23982 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.581889 23975 ts_tablet_manager.cc:1434] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:12:43.582626 23982 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.583081 23975 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.589324 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:43.591921 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 102493ff11494b009f79963bb57be511 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:43.592259 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:43.593075 23749 leader_election.cc:343] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:43.593808 23751 leader_election.cc:343] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.594198 23975 tablet_bootstrap.cc:654] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.594236 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:43.595743 23982 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:43.606343 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102493ff11494b009f79963bb57be511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.607092 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.607774 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:43.609308 23899 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:43.610145 23983 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:43.610560 23983 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.610203 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 3eeb67716b0e4f49aa1a4bd5815f684b (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:43.610890 23983 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.611521 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eeb67716b0e4f49aa1a4bd5815f684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.616035 23850 tablet_service.cc:1511] Processing CreateTablet for tablet 102493ff11494b009f79963bb57be511 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:43.617326 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102493ff11494b009f79963bb57be511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.617748 23977 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:43.618216 23977 ts_tablet_manager.cc:1403] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.002s
I20260709 12:12:43.618433 23916 tablet_service.cc:1511] Processing CreateTablet for tablet d6d3d4f634284532bbf6ad7d4ac19214 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:43.619699 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6d3d4f634284532bbf6ad7d4ac19214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.621012 23977 raft_consensus.cc:359] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.621716 23977 raft_consensus.cc:385] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.621985 23977 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.622269 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 3eeb67716b0e4f49aa1a4bd5815f684b (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:43.623564 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3eeb67716b0e4f49aa1a4bd5815f684b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.629457 23983 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.631165 23976 tablet_bootstrap.cc:654] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.632457 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:43.632506 23977 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.633242 23785 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.638235 23767 tablet_service.cc:1511] Processing CreateTablet for tablet ed3c66ee31004577b517e2c4322696b6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:43.640537 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3c66ee31004577b517e2c4322696b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.640220 23844 tablet_service.cc:1511] Processing CreateTablet for tablet ed3c66ee31004577b517e2c4322696b6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:43.641539 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3c66ee31004577b517e2c4322696b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.644330 23983 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.653457 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:43.655493 23899 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:43.656355 23983 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:43.659154 23977 ts_tablet_manager.cc:1434] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.003s
I20260709 12:12:43.667937 23977 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:43.668401 23770 tablet_service.cc:1511] Processing CreateTablet for tablet a979dd022fc24afd8765a3cdb5f7fab3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:43.669741 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a979dd022fc24afd8765a3cdb5f7fab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.674932 23981 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.675457 23981 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.677599 23981 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.678355 23919 tablet_service.cc:1511] Processing CreateTablet for tablet 102493ff11494b009f79963bb57be511 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:43.679523 23975 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.679816 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102493ff11494b009f79963bb57be511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.680043 23975 ts_tablet_manager.cc:1403] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.017s
I20260709 12:12:43.682070 23976 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:43.682556 23976 ts_tablet_manager.cc:1403] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.129s	user 0.011s	sys 0.003s
I20260709 12:12:43.685019 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:43.685540 23976 raft_consensus.cc:359] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.686260 23976 raft_consensus.cc:385] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.686558 23976 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.687660 23976 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.687917 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:43.688946 23824 leader_election.cc:343] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.683014 23975 raft_consensus.cc:359] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:43.694550 23825 leader_election.cc:343] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.695003 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:43.695628 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:43.696064 23981 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:43.693624 23975 raft_consensus.cc:385] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.696868 23975 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.697031 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:43.697654 23975 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.689222 23977 tablet_bootstrap.cc:654] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.700258 23975 ts_tablet_manager.cc:1434] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 12:12:43.700600 23916 tablet_service.cc:1511] Processing CreateTablet for tablet ed3c66ee31004577b517e2c4322696b6 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:43.701275 23975 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.703294 23848 tablet_service.cc:1511] Processing CreateTablet for tablet a979dd022fc24afd8765a3cdb5f7fab3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:43.701920 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3c66ee31004577b517e2c4322696b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.708071 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a979dd022fc24afd8765a3cdb5f7fab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.709847 23845 tablet_service.cc:1511] Processing CreateTablet for tablet e943fd8cd04e4a5e9678a4bdbd7b1449 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:12:43.711144 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943fd8cd04e4a5e9678a4bdbd7b1449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.705075 23983 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:43.735728 23918 tablet_service.cc:1511] Processing CreateTablet for tablet a979dd022fc24afd8765a3cdb5f7fab3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:43.736676 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 055432f1640d4d5a8a4ec2355e0b89ac (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:43.737061 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a979dd022fc24afd8765a3cdb5f7fab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.737962 23767 tablet_service.cc:1511] Processing CreateTablet for tablet e943fd8cd04e4a5e9678a4bdbd7b1449 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:12:43.738487 23900 leader_election.cc:343] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.739151 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943fd8cd04e4a5e9678a4bdbd7b1449. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:43.739630 23900 leader_election.cc:343] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.741859 23924 tablet_service.cc:1511] Processing CreateTablet for tablet e943fd8cd04e4a5e9678a4bdbd7b1449 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:12:43.743147 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e943fd8cd04e4a5e9678a4bdbd7b1449. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.744813 23983 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.748067 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055432f1640d4d5a8a4ec2355e0b89ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.748196 23977 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:43.748579 23975 tablet_bootstrap.cc:654] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.748723 23977 ts_tablet_manager.cc:1403] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.000s
I20260709 12:12:43.757030 23977 raft_consensus.cc:359] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.759792 23976 ts_tablet_manager.cc:1434] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.022s
I20260709 12:12:43.760932 23976 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:43.761003 23844 tablet_service.cc:1511] Processing CreateTablet for tablet 055432f1640d4d5a8a4ec2355e0b89ac (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:43.762671 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055432f1640d4d5a8a4ec2355e0b89ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.758019 23977 raft_consensus.cc:385] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.768736 23916 tablet_service.cc:1511] Processing CreateTablet for tablet 055432f1640d4d5a8a4ec2355e0b89ac (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:43.770284 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055432f1640d4d5a8a4ec2355e0b89ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.769150 23977 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.784293 23682 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:43.789860 23975 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.790488 23975 ts_tablet_manager.cc:1403] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.006s
I20260709 12:12:43.780067 23977 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.793282 23977 ts_tablet_manager.cc:1434] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:12:43.793210 23975 raft_consensus.cc:359] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.794023 23975 raft_consensus.cc:385] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.794317 23975 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.795035 23975 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.797801 23975 ts_tablet_manager.cc:1434] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:43.799008 23975 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.799963 23773 tablet_service.cc:1511] Processing CreateTablet for tablet d9f2f570298740c691fa19d449b3fa4f (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:43.800788 23976 tablet_bootstrap.cc:654] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.801276 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f2f570298740c691fa19d449b3fa4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.805411 23975 tablet_bootstrap.cc:654] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.809109 23977 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:43.812206 23981 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.812613 23976 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:43.812768 23981 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.813542 23976 ts_tablet_manager.cc:1403] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.001s
I20260709 12:12:43.815120 23981 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.817400 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.817554 23976 raft_consensus.cc:359] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.818202 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:43.819496 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:43.821353 23824 leader_election.cc:343] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.822683 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:43.823632 23981 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:43.824008 23981 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.824355 23981 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.827879 23919 tablet_service.cc:1511] Processing CreateTablet for tablet d9f2f570298740c691fa19d449b3fa4f (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:43.829237 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f2f570298740c691fa19d449b3fa4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.818336 23976 raft_consensus.cc:385] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.830684 23976 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.831698 23976 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.833211 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9d6097cb8d40efa55ff97de5b26130 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:43.838313 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9d6097cb8d40efa55ff97de5b26130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.841040 23848 tablet_service.cc:1511] Processing CreateTablet for tablet a1b618b31b2e427b8c073b2c24771903 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:43.842327 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b618b31b2e427b8c073b2c24771903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.831271 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9d6097cb8d40efa55ff97de5b26130 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:43.830545 23844 tablet_service.cc:1511] Processing CreateTablet for tablet d9f2f570298740c691fa19d449b3fa4f (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:43.851791 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9d6097cb8d40efa55ff97de5b26130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.852644 23924 tablet_service.cc:1511] Processing CreateTablet for tablet a1b618b31b2e427b8c073b2c24771903 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:43.852653 23850 tablet_service.cc:1511] Processing CreateTablet for tablet 34e5e0c10245477d9010ece72d08ae51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:43.867513 23915 tablet_service.cc:1511] Processing CreateTablet for tablet 8f44bb7910714c46af4e3d0e2076e7c5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:43.853734 23921 tablet_service.cc:1511] Processing CreateTablet for tablet 34e5e0c10245477d9010ece72d08ae51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:43.848992 23767 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9d6097cb8d40efa55ff97de5b26130 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:43.872942 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e5e0c10245477d9010ece72d08ae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.873454 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9d6097cb8d40efa55ff97de5b26130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.873342 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e5e0c10245477d9010ece72d08ae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.857105 23983 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.874656 23983 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.857167 23770 tablet_service.cc:1511] Processing CreateTablet for tablet a1b618b31b2e427b8c073b2c24771903 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:43.858858 23766 tablet_service.cc:1511] Processing CreateTablet for tablet 34e5e0c10245477d9010ece72d08ae51 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:43.860484 23771 tablet_service.cc:1511] Processing CreateTablet for tablet 5038ac294ceb4758b5b5de70f2469d5d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:43.862040 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 5038ac294ceb4758b5b5de70f2469d5d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:43.869930 24003 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.879515 24003 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.881146 23983 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.881382 23976 ts_tablet_manager.cc:1434] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:12:43.881713 24003 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.883416 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34e5e0c10245477d9010ece72d08ae51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.885588 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.864143 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 8f44bb7910714c46af4e3d0e2076e7c5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:43.862043 23847 tablet_service.cc:1511] Processing CreateTablet for tablet 5038ac294ceb4758b5b5de70f2469d5d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:43.877589 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b618b31b2e427b8c073b2c24771903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.888640 23976 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:43.889999 23983 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.890494 23983 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.891894 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5038ac294ceb4758b5b5de70f2469d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.892154 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:43.888584 23899 leader_election.cc:343] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:43.894117 23899 leader_election.cc:343] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.895258 23977 tablet_bootstrap.cc:654] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.886211 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b618b31b2e427b8c073b2c24771903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.865341 23842 tablet_service.cc:1511] Processing CreateTablet for tablet 8f44bb7910714c46af4e3d0e2076e7c5 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:43.896225 23771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5038ac294ceb4758b5b5de70f2469d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.896773 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f44bb7910714c46af4e3d0e2076e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.899152 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.899905 23785 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:43.899451 23975 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:43.902710 23975 ts_tablet_manager.cc:1403] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.001s
I20260709 12:12:43.902004 23983 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.905155 23915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f44bb7910714c46af4e3d0e2076e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.907961 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:43.908452 23983 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.908705 23858 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:43.910116 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:43.910977 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:43.911742 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:43.912555 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f44bb7910714c46af4e3d0e2076e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.912439 23860 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:43.912760 24003 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:43.913112 24003 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.913434 24003 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.916152 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5038ac294ceb4758b5b5de70f2469d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.916558 23975 raft_consensus.cc:359] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.917337 23975 raft_consensus.cc:385] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:43.917637 23975 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:43.918427 23975 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:43.920060 23900 leader_election.cc:343] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.920593 23900 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:43.921482 24002 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:43.922767 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:43.928012 24002 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:43.928388 24002 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.928678 24002 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.932188 23981 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.935261 23981 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.935832 23981 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.910948 23983 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.939410 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.940121 23784 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:43.919996 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f2f570298740c691fa19d449b3fa4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.949952 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:43.936249 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:43.956841 23785 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.957567 23975 ts_tablet_manager.cc:1434] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.003s
I20260709 12:12:43.936273 23981 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.959975 23981 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.960512 23768 tablet_service.cc:1511] Processing CreateTablet for tablet e76b9730b35144689e48d4d145fabb18 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:12:43.961836 23768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76b9730b35144689e48d4d145fabb18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.963356 23976 tablet_bootstrap.cc:654] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:43.963990 23983 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.964555 23983 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:43.964895 23983 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.965221 23983 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.936301 24002 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.971629 24002 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:43.975310 23975 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:43.978873 23982 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.978844 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:43.979686 23982 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:43.980059 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:43.980618 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:43.980777 23784 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:43.981223 23859 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:43.981765 23982 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:43.984059 23769 tablet_service.cc:1511] Processing CreateTablet for tablet 7f5f207bbfed4f119db8464be0a5a8cf (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 12:12:43.984685 23900 leader_election.cc:343] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.985361 23769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f5f207bbfed4f119db8464be0a5a8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:43.989391 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:43.977888 23844 tablet_service.cc:1511] Processing CreateTablet for tablet e76b9730b35144689e48d4d145fabb18 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:12:43.990185 23845 tablet_service.cc:1511] Processing CreateTablet for tablet 7f5f207bbfed4f119db8464be0a5a8cf (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:12:44.000612 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.002800 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:44.004338 23749 leader_election.cc:343] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:43.999449 23977 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.006172 23977 ts_tablet_manager.cc:1403] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.197s	user 0.016s	sys 0.025s
I20260709 12:12:44.005738 23975 tablet_bootstrap.cc:654] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.009055 23977 raft_consensus.cc:359] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.009802 23977 raft_consensus.cc:385] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.010097 23977 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.010838 23977 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:43.991878 23981 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.013304 23981 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.983099 23919 tablet_service.cc:1511] Processing CreateTablet for tablet e76b9730b35144689e48d4d145fabb18 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
W20260709 12:12:44.017038 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68596.2us
I20260709 12:12:44.020818 23977 ts_tablet_manager.cc:1434] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:12:44.020913 23921 tablet_service.cc:1511] Processing CreateTablet for tablet 7f5f207bbfed4f119db8464be0a5a8cf (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:12:44.037464 23976 tablet_bootstrap.cc:492] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.021572 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.005693 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:44.039208 23923 tablet_service.cc:1511] Processing CreateTablet for tablet 872245a7005f472bb7f6cfafda680b74 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:12:44.035725 23975 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.022250 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.022823 23770 tablet_service.cc:1511] Processing CreateTablet for tablet 956a6729b1f34c499670aa77b5896a53 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:44.023362 23844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76b9730b35144689e48d4d145fabb18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.023449 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.024761 23916 tablet_service.cc:1511] Processing CreateTablet for tablet 956a6729b1f34c499670aa77b5896a53 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:44.026371 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 872245a7005f472bb7f6cfafda680b74 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:12:44.047236 23860 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:44.037926 23921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f5f207bbfed4f119db8464be0a5a8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.012809 24003 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.030637 23983 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.031651 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.051717 23919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76b9730b35144689e48d4d145fabb18. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:44.052604 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:44.037988 23976 ts_tablet_manager.cc:1403] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.150s	user 0.007s	sys 0.006s
I20260709 12:12:44.052722 24003 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:44.053891 23899 leader_election.cc:343] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.053923 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:44.054962 23860 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.044492 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956a6729b1f34c499670aa77b5896a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.048403 23767 tablet_service.cc:1511] Processing CreateTablet for tablet e2366a1e0cb342ee891c776affbfbca3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:44.019212 23981 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.057741 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956a6729b1f34c499670aa77b5896a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.058228 23767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2366a1e0cb342ee891c776affbfbca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.049640 23850 tablet_service.cc:1511] Processing CreateTablet for tablet 956a6729b1f34c499670aa77b5896a53 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:44.060420 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872245a7005f472bb7f6cfafda680b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.040930 23975 ts_tablet_manager.cc:1403] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.066s	user 0.007s	sys 0.009s
I20260709 12:12:44.062459 23976 raft_consensus.cc:359] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.063202 23976 raft_consensus.cc:385] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:12:44.063696 23824 leader_election.cc:343] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.063832 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.064360 23785 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.064556 23848 tablet_service.cc:1511] Processing CreateTablet for tablet 872245a7005f472bb7f6cfafda680b74 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:12:44.064944 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.066150 23982 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.066509 23982 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.066864 23982 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.066617 23849 tablet_service.cc:1511] Processing CreateTablet for tablet e2366a1e0cb342ee891c776affbfbca3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:44.067782 24013 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.080006 23845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f5f207bbfed4f119db8464be0a5a8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.080698 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2366a1e0cb342ee891c776affbfbca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.081311 23850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956a6729b1f34c499670aa77b5896a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.082281 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.082922 23784 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.083137 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872245a7005f472bb7f6cfafda680b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.103621 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:12:44.105222 23900 leader_election.cc:343] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.063473 23976 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.111634 23860 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.108292 23983 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.113515 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:44.114305 23924 tablet_service.cc:1511] Processing CreateTablet for tablet e2366a1e0cb342ee891c776affbfbca3 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:44.121321 23983 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.051777 23859 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.122025 23983 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:44.123451 23975 raft_consensus.cc:359] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:44.065055 23824 leader_election.cc:343] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.124361 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:44.130272 24003 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.130800 24003 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:44.123023 23983 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.119716 23923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872245a7005f472bb7f6cfafda680b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.131750 24003 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.143013 23982 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:44.132637 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 67615.6us
I20260709 12:12:44.136379 23977 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.155272 23785 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.138738 23784 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.124203 23975 raft_consensus.cc:385] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.145148 23981 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.169164 23770 tablet_service.cc:1511] Processing CreateTablet for tablet 349b687db5c44196bafc207c26ca3679 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:44.171130 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 349b687db5c44196bafc207c26ca3679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.174767 23981 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.176412 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.158764 24013 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:44.169575 24018 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.177609 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:44.186372 24014 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.186265 23915 tablet_service.cc:1511] Processing CreateTablet for tablet 349b687db5c44196bafc207c26ca3679 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:44.186975 24014 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:44.161051 23975 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
W20260709 12:12:44.187815 23824 leader_election.cc:343] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.187964 24014 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.189002 23975 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.189193 23766 tablet_service.cc:1511] Processing CreateTablet for tablet d79b6f8bf5274da7a3f667b9988c5a84 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:44.190668 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79b6f8bf5274da7a3f667b9988c5a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.191454 24018 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.194199 23977 tablet_bootstrap.cc:654] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.181103 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:44.196646 23859 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.198608 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:44.200212 23824 leader_election.cc:343] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.201285 24018 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:44.202044 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.202699 23784 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.208838 23842 tablet_service.cc:1511] Processing CreateTablet for tablet 349b687db5c44196bafc207c26ca3679 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:44.210474 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 349b687db5c44196bafc207c26ca3679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.178751 23976 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.211802 23784 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:44.213466 23976 ts_tablet_manager.cc:1434] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.154s	user 0.006s	sys 0.000s
I20260709 12:12:44.220007 23976 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.214753 23982 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.216101 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.222551 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.217979 23916 tablet_service.cc:1511] Processing CreateTablet for tablet d79b6f8bf5274da7a3f667b9988c5a84 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:44.228441 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.228550 23848 tablet_service.cc:1511] Processing CreateTablet for tablet d79b6f8bf5274da7a3f667b9988c5a84 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:44.229743 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79b6f8bf5274da7a3f667b9988c5a84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.229945 24002 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.230410 24002 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:44.228361 23749 leader_election.cc:343] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.214408 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2366a1e0cb342ee891c776affbfbca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.220119 23773 tablet_service.cc:1511] Processing CreateTablet for tablet b7caafee6779404494f8df62d888c5ef (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:44.233741 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7caafee6779404494f8df62d888c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.235526 24002 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.237437 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.238200 23785 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:44.242174 23847 tablet_service.cc:1511] Processing CreateTablet for tablet b7caafee6779404494f8df62d888c5ef (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:44.243618 23847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7caafee6779404494f8df62d888c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.243860 23918 tablet_service.cc:1511] Processing CreateTablet for tablet b7caafee6779404494f8df62d888c5ef (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:44.244997 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7caafee6779404494f8df62d888c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.255848 24013 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.256513 24018 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.257351 24018 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:44.258222 24018 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.259541 23915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 349b687db5c44196bafc207c26ca3679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.268445 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79b6f8bf5274da7a3f667b9988c5a84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:44.259891 23825 leader_election.cc:343] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.302762 23975 ts_tablet_manager.cc:1434] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.241s	user 0.013s	sys 0.002s
I20260709 12:12:44.306521 23975 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.293234 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.308106 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:44.308991 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54567.9us
I20260709 12:12:44.309466 24019 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.318754 23983 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.324834 23682 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.328748 23770 tablet_service.cc:1511] Processing CreateTablet for tablet bec362a6d01840d39be46f0c3d28c218 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:12:44.330029 23770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec362a6d01840d39be46f0c3d28c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.316700 23859 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:44.332115 23773 tablet_service.cc:1511] Processing CreateTablet for tablet 053977846bfa4adf9703ea669a81f41d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:44.333309 23773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053977846bfa4adf9703ea669a81f41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.337703 23848 tablet_service.cc:1511] Processing CreateTablet for tablet bec362a6d01840d39be46f0c3d28c218 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:12:44.339046 23848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec362a6d01840d39be46f0c3d28c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.339346 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.340587 24021 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.341029 24021 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:44.341809 24021 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.344955 23766 tablet_service.cc:1511] Processing CreateTablet for tablet ef461146a2924dba8b8e2906d76859f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:12:44.346253 23766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef461146a2924dba8b8e2906d76859f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.347879 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:44.348717 23983 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.350517 23918 tablet_service.cc:1511] Processing CreateTablet for tablet 053977846bfa4adf9703ea669a81f41d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:44.351703 23918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053977846bfa4adf9703ea669a81f41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.351420 23924 tablet_service.cc:1511] Processing CreateTablet for tablet bec362a6d01840d39be46f0c3d28c218 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:12:44.353426 23924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec362a6d01840d39be46f0c3d28c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.356088 23681 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:44.358388 23983 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.357900 23681 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:44.358474 23916 tablet_service.cc:1511] Processing CreateTablet for tablet ef461146a2924dba8b8e2906d76859f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 12:12:44.367515 23900 consensus_peers.cc:597] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:44.368578 23916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef461146a2924dba8b8e2906d76859f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.372429 23842 tablet_service.cc:1511] Processing CreateTablet for tablet ef461146a2924dba8b8e2906d76859f1 (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:12:44.372767 23849 tablet_service.cc:1511] Processing CreateTablet for tablet 053977846bfa4adf9703ea669a81f41d (DEFAULT_TABLE table=test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:44.373878 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef461146a2924dba8b8e2906d76859f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.377381 23977 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.377903 23977 ts_tablet_manager.cc:1403] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.253s	user 0.006s	sys 0.014s
I20260709 12:12:44.381836 23976 tablet_bootstrap.cc:654] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.377800 23681 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:44.384676 23975 tablet_bootstrap.cc:654] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.421566 23977 raft_consensus.cc:359] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.422448 23977 raft_consensus.cc:385] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.422802 23977 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.423729 23977 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.429505 23977 ts_tablet_manager.cc:1434] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 12:12:44.433319 23977 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.412521 23849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053977846bfa4adf9703ea669a81f41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:44.434651 23682 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.460615 23683 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.439572 24021 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.473323 23977 tablet_bootstrap.cc:654] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.472270 24021 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.484995 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.486418 24018 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.487165 24021 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.487000 24018 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.487684 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:44.489070 23751 leader_election.cc:343] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.491091 24018 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:44.491014 23749 leader_election.cc:343] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.492183 23977 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.492293 23975 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.492821 23977 ts_tablet_manager.cc:1403] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.004s
I20260709 12:12:44.492856 23975 ts_tablet_manager.cc:1403] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.187s	user 0.020s	sys 0.040s
I20260709 12:12:44.493383 23749 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.493566 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.492183 23976 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.494385 23976 ts_tablet_manager.cc:1403] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.275s	user 0.023s	sys 0.011s
I20260709 12:12:44.495026 23933 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:44.495234 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.496361 24021 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.496394 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:44.497597 24018 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.497959 24018 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.498327 24018 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.501693 23977 raft_consensus.cc:359] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.498100 23975 raft_consensus.cc:359] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.502429 23977 raft_consensus.cc:385] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.502647 23975 raft_consensus.cc:385] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.502743 23977 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.503070 23975 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.503556 23977 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.503834 23975 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.504568 23976 raft_consensus.cc:359] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.507792 23976 raft_consensus.cc:385] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.505610 23977 ts_tablet_manager.cc:1434] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:12:44.506011 23785 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:44.509996 23977 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.516119 24018 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.518569 24018 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.519366 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.515467 23977 tablet_bootstrap.cc:654] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.519734 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.520366 23933 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.508140 23976 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.522590 23976 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.507452 23975 ts_tablet_manager.cc:1434] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:12:44.525131 23975 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.526525 23785 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.528398 23933 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:44.529698 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:44.530618 24018 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.531147 24018 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:44.531890 23976 ts_tablet_manager.cc:1434] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 12:12:44.532011 24018 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.537320 24003 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.537940 23976 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.541226 23977 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.533353 23975 tablet_bootstrap.cc:654] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.541709 23977 ts_tablet_manager.cc:1403] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.032s	user 0.018s	sys 0.003s
I20260709 12:12:44.546041 23785 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:44.547644 23976 tablet_bootstrap.cc:654] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.554188 23977 raft_consensus.cc:359] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.554970 23977 raft_consensus.cc:385] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.555246 23977 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.556097 23977 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.556799 23682 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.558121 23977 ts_tablet_manager.cc:1434] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:12:44.559028 23977 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.564826 23977 tablet_bootstrap.cc:654] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.568280 23976 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.568815 23976 ts_tablet_manager.cc:1403] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.031s	user 0.023s	sys 0.004s
I20260709 12:12:44.571627 23976 raft_consensus.cc:359] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.572418 23976 raft_consensus.cc:385] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.572750 23976 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.573434 23976 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.575408 23976 ts_tablet_manager.cc:1434] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:44.576398 23976 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.576894 24002 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.582320 23977 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.584080 23977 ts_tablet_manager.cc:1403] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.011s
I20260709 12:12:44.585204 23975 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.585668 23975 ts_tablet_manager.cc:1403] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.008s
I20260709 12:12:44.586244 23977 raft_consensus.cc:359] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.586750 23977 raft_consensus.cc:385] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.586970 23977 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.587556 23977 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.588325 23975 raft_consensus.cc:359] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.588981 23975 raft_consensus.cc:385] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.589306 23975 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.589915 23977 ts_tablet_manager.cc:1434] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:12:44.590185 23975 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.590880 23977 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.592192 23975 ts_tablet_manager.cc:1434] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:12:44.593154 23975 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.595708 23977 tablet_bootstrap.cc:654] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.598727 23975 tablet_bootstrap.cc:654] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.601505 23976 tablet_bootstrap.cc:654] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.606282 23977 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.606758 23977 ts_tablet_manager.cc:1403] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 12:12:44.612432 23977 raft_consensus.cc:359] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.613121 23977 raft_consensus.cc:385] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.613399 23977 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.614202 23977 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.615222 23975 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.615669 23976 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.615705 23975 ts_tablet_manager.cc:1403] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.004s
I20260709 12:12:44.616160 23977 ts_tablet_manager.cc:1434] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:44.616410 23976 ts_tablet_manager.cc:1403] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 12:12:44.617518 23977 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.618626 23975 raft_consensus.cc:359] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.619306 23975 raft_consensus.cc:385] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.619071 23976 raft_consensus.cc:359] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.619695 23975 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.620040 23976 raft_consensus.cc:385] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.620404 23976 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.620417 23975 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.621531 23976 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.622910 23975 ts_tablet_manager.cc:1434] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:44.623807 23977 tablet_bootstrap.cc:654] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.625391 24018 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.625905 24018 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.629012 24018 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.632874 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.637480 23783 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.630685 23975 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.630503 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.639815 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.631112 24002 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.641050 24018 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.641077 24002 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.644100 24002 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.644876 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.645637 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.647269 23859 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:44.647382 23975 tablet_bootstrap.cc:654] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.648900 23900 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:44.649834 24002 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.652210 24018 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.652907 24018 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.656556 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.657609 23784 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:44.659160 24002 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.660523 24018 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.661329 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.663372 23824 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.665014 23976 ts_tablet_manager.cc:1434] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.001s
I20260709 12:12:44.666237 23976 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.667799 24018 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.675798 24002 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.677834 24013 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:44.687076 24018 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.690934 23976 tablet_bootstrap.cc:654] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.694885 23975 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.695410 23975 ts_tablet_manager.cc:1403] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.065s	user 0.025s	sys 0.009s
I20260709 12:12:44.696376 23977 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.696771 23977 ts_tablet_manager.cc:1403] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.079s	user 0.039s	sys 0.035s
I20260709 12:12:44.698123 23975 raft_consensus.cc:359] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.698859 23975 raft_consensus.cc:385] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.698657 23977 raft_consensus.cc:359] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.699162 23975 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.699323 23977 raft_consensus.cc:385] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.699654 23977 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.700016 23975 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.700300 23977 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.702499 23977 ts_tablet_manager.cc:1434] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:44.702546 23975 ts_tablet_manager.cc:1434] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:12:44.703715 24002 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.704053 23977 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.710054 23977 tablet_bootstrap.cc:654] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.712720 23975 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.715667 24003 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.721539 23975 tablet_bootstrap.cc:654] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.726004 24003 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.731881 24014 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.732372 24014 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.734853 24014 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.736387 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.738760 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.739768 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:44.741762 23899 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:44.748956 24018 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:44.750614 24021 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.754366 23983 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.764796 23983 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.778049 23976 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.778053 24057 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:44.778687 23976 ts_tablet_manager.cc:1403] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.012s
I20260709 12:12:44.781492 23976 raft_consensus.cc:359] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.782908 23976 raft_consensus.cc:385] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.783263 23976 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.783332 23981 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:44.784366 23976 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.790443 23976 ts_tablet_manager.cc:1434] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:12:44.792912 24014 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.793643 24014 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.799113 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.800041 23859 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:44.802668 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.803169 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:44.804029 23899 leader_election.cc:343] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.804131 23983 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.804618 23983 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.805231 23983 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.808291 23860 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.813261 23983 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.814127 23976 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.816032 24018 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:44.814411 24014 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:44.826526 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:44.827404 23859 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.835167 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:12:44.837036 23899 leader_election.cc:343] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.837741 23983 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:44.840209 23976 tablet_bootstrap.cc:654] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.855626 23981 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.856204 23981 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.858520 23981 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.860101 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.861059 23785 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:44.861967 23975 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:44.862663 23975 ts_tablet_manager.cc:1403] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.150s	user 0.042s	sys 0.028s
I20260709 12:12:44.862847 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.865249 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:12:44.865875 23824 leader_election.cc:343] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.854239 23859 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.866142 23981 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.866636 23981 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.866978 23981 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.870898 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:44.872401 23983 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.872960 23983 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:44.873875 23983 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:44.889104 24057 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.887980 23975 raft_consensus.cc:359] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.889698 24057 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.892777 24057 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.893666 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.894735 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:44.895097 23751 leader_election.cc:343] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.895849 23975 raft_consensus.cc:385] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.896193 23975 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.897436 23975 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:44.898609 23749 leader_election.cc:343] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.899170 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.900102 24021 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.903204 24018 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.903762 24018 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.907095 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.908320 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:44.908636 23824 leader_election.cc:343] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.911088 23975 ts_tablet_manager.cc:1434] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 12:12:44.902922 23981 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:44.910405 23825 leader_election.cc:343] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.907073 24018 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.912667 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.914002 24019 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.915091 23975 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:44.916074 24003 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.916366 23976 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:44.916792 23976 ts_tablet_manager.cc:1403] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.125s	user 0.004s	sys 0.014s
I20260709 12:12:44.916585 24003 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.918746 23981 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.919512 24021 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.919477 23976 raft_consensus.cc:359] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.920279 23976 raft_consensus.cc:385] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.920595 23976 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.918730 24003 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.921420 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.921377 23976 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.922107 23859 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:44.921437 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.923174 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:44.923664 23976 ts_tablet_manager.cc:1434] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:44.924175 24003 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:12:44.924615 23824 leader_election.cc:343] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.924588 23976 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:44.920106 24021 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.925450 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.926023 23785 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.927207 24021 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.924528 24003 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.929108 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:44.929414 24003 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.929101 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.929877 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:44.929812 23933 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:44.929097 23682 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.930629 23783 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:44.932441 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:44.936470 23977 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:44.937166 23785 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:44.938304 23977 ts_tablet_manager.cc:1403] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.235s	user 0.067s	sys 0.108s
I20260709 12:12:44.938253 24021 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:44.939349 24021 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:44.940843 23751 leader_election.cc:343] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.941735 24021 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.945524 23976 tablet_bootstrap.cc:654] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.945631 24003 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.948786 24021 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.952361 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.954311 24003 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:44.955050 23977 raft_consensus.cc:359] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.956112 23977 raft_consensus.cc:385] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:44.956485 23977 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:44.956835 24021 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.957701 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:44.957811 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:44.958351 23860 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.958935 24019 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:12:44.959062 23751 leader_election.cc:343] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.959410 24019 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:44.958087 23977 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.960299 24019 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.961956 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:44.962517 23933 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.962759 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:44.963299 23785 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:44.965376 23977 ts_tablet_manager.cc:1434] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.001s
I20260709 12:12:44.966379 23977 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:44.968199 24013 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.968762 24013 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.971168 24013 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:44.972262 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:44.973640 23824 leader_election.cc:343] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.975785 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:44.976773 23860 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.977128 23933 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:44.977270 23785 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:44.978030 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:44.978571 23825 leader_election.cc:343] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:44.979081 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:44.980482 24003 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.980952 24013 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:44.980116 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:44.983356 24021 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:44.983577 24003 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:44.984375 24021 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:44.985780 24003 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.987931 23977 tablet_bootstrap.cc:654] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:44.991969 24021 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:44.992594 23981 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:44.995518 23682 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:44.997246 23975 tablet_bootstrap.cc:654] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.013753 23976 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.014223 23976 ts_tablet_manager.cc:1403] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.000s
I20260709 12:12:45.016939 23976 raft_consensus.cc:359] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.017622 23976 raft_consensus.cc:385] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.017138 23682 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:45.017899 23976 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.018587 23976 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.019707 23681 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:45.020823 23976 ts_tablet_manager.cc:1434] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:12:45.021837 23976 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.028357 24018 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:45.039028 23977 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.039574 23977 ts_tablet_manager.cc:1403] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.073s	user 0.004s	sys 0.012s
I20260709 12:12:45.041143 23976 tablet_bootstrap.cc:654] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.043000 23977 raft_consensus.cc:359] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.043831 23977 raft_consensus.cc:385] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.044117 23977 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.044914 23977 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.047197 23977 ts_tablet_manager.cc:1434] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:12:45.048229 23977 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.055222 23977 tablet_bootstrap.cc:654] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.067035 23977 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.067633 23977 ts_tablet_manager.cc:1403] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:12:45.070648 23977 raft_consensus.cc:359] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.072186 23977 raft_consensus.cc:385] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.072522 23977 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.073109 23977 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.074738 23975 tablet_bootstrap.cc:492] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.075259 23975 ts_tablet_manager.cc:1403] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.160s	user 0.029s	sys 0.040s
I20260709 12:12:45.078001 23975 raft_consensus.cc:359] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:45.078745 23975 raft_consensus.cc:385] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.079061 23975 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.079871 23975 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:45.085995 23977 ts_tablet_manager.cc:1434] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:12:45.086964 23976 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.088083 23976 ts_tablet_manager.cc:1403] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.014s
I20260709 12:12:45.090679 23976 raft_consensus.cc:359] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.091372 23976 raft_consensus.cc:385] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.091666 23976 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.087426 23975 ts_tablet_manager.cc:1434] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:12:45.092062 23977 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.092938 23975 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:12:45.093173 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:45.098177 23975 tablet_bootstrap.cc:654] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.099005 23977 tablet_bootstrap.cc:654] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.103168 23975 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.103670 23975 ts_tablet_manager.cc:1403] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 12:12:45.106357 23975 raft_consensus.cc:359] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.107076 23975 raft_consensus.cc:385] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.107342 23975 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.108623 23977 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.108110 23975 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.109228 23976 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.110847 23975 ts_tablet_manager.cc:1434] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:45.111377 23976 ts_tablet_manager.cc:1434] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.023s	user 0.000s	sys 0.008s
I20260709 12:12:45.112188 23975 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.112232 23977 ts_tablet_manager.cc:1403] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:12:45.115319 23977 raft_consensus.cc:359] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.116050 23977 raft_consensus.cc:385] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.116333 23977 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.116729 23976 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.117051 23977 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.119107 23975 tablet_bootstrap.cc:654] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.122835 23977 ts_tablet_manager.cc:1434] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:12:45.124028 23977 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.125114 23976 tablet_bootstrap.cc:654] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.130561 23977 tablet_bootstrap.cc:654] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.143287 23977 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.143913 23977 ts_tablet_manager.cc:1403] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:12:45.146811 23977 raft_consensus.cc:359] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.147581 23977 raft_consensus.cc:385] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.147907 23977 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.149093 23977 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.151748 23977 ts_tablet_manager.cc:1434] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:45.156394 23977 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.162984 23977 tablet_bootstrap.cc:654] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.178781 23976 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.179191 23976 ts_tablet_manager.cc:1403] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.041s
I20260709 12:12:45.179250 23975 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.179718 23975 ts_tablet_manager.cc:1403] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.068s	user 0.028s	sys 0.031s
I20260709 12:12:45.180918 23977 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.181437 23977 ts_tablet_manager.cc:1403] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.015s
I20260709 12:12:45.182132 23976 raft_consensus.cc:359] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.182263 23975 raft_consensus.cc:359] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.182853 23976 raft_consensus.cc:385] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.182931 23975 raft_consensus.cc:385] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.183248 23975 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.183203 23976 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.183969 23975 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.184144 23976 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.184306 23977 raft_consensus.cc:359] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.185894 23975 ts_tablet_manager.cc:1434] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:45.186517 23976 ts_tablet_manager.cc:1434] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:45.186501 23977 raft_consensus.cc:385] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.187048 23977 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.188134 23976 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.188169 23977 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.191306 23975 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.194625 23977 ts_tablet_manager.cc:1434] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:12:45.195503 23977 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.201406 23976 tablet_bootstrap.cc:654] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.201499 23975 tablet_bootstrap.cc:654] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.205008 23977 tablet_bootstrap.cc:654] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.223294 24021 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.223953 24021 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:45.224413 23975 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.224885 23975 ts_tablet_manager.cc:1403] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 12:12:45.226096 24021 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.226846 23977 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.226984 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.227362 23977 ts_tablet_manager.cc:1403] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.015s
I20260709 12:12:45.227864 23860 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:45.227500 23975 raft_consensus.cc:359] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.228252 23975 raft_consensus.cc:385] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.228539 23975 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.228930 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.229192 23975 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.230345 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:45.231164 23975 ts_tablet_manager.cc:1434] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:12:45.231448 24057 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.232426 23977 raft_consensus.cc:359] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.233135 23977 raft_consensus.cc:385] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.233460 23977 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.234191 23977 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.236155 23977 ts_tablet_manager.cc:1434] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:12:45.237254 23977 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.241945 23977 tablet_bootstrap.cc:654] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.244201 23975 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.247684 24057 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:45.249800 23976 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.250288 23976 ts_tablet_manager.cc:1403] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.009s
I20260709 12:12:45.254858 23975 tablet_bootstrap.cc:654] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.257151 23976 raft_consensus.cc:359] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.257885 23976 raft_consensus.cc:385] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.258148 23976 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.258919 23976 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.261183 23976 ts_tablet_manager.cc:1434] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:12:45.261945 23976 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.263867 23975 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.264406 23975 ts_tablet_manager.cc:1403] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.020s	user 0.002s	sys 0.012s
I20260709 12:12:45.265491 23977 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.265942 23977 ts_tablet_manager.cc:1403] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.006s
I20260709 12:12:45.267181 23976 tablet_bootstrap.cc:654] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.267767 23975 raft_consensus.cc:359] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.268508 23975 raft_consensus.cc:385] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.268867 23975 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.268649 23977 raft_consensus.cc:359] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.269738 23975 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.271735 23977 raft_consensus.cc:385] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.272068 23977 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.273023 23976 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.272923 23977 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.275288 23975 ts_tablet_manager.cc:1434] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:12:45.273496 23976 ts_tablet_manager.cc:1403] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:12:45.279666 23977 ts_tablet_manager.cc:1434] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:12:45.281790 23975 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.281955 23977 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.285598 24002 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.286162 24002 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.288057 23975 tablet_bootstrap.cc:654] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.288285 24002 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.290107 23976 raft_consensus.cc:359] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.293148 23976 raft_consensus.cc:385] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.293462 23976 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.291438 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.294240 23976 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.295122 23860 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:45.296487 23976 ts_tablet_manager.cc:1434] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 12:12:45.296669 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.292291 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.297508 23785 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:45.297813 23976 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.297816 24003 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.298553 24003 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.298916 24003 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.298087 23983 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:12:45.301882 23977 tablet_bootstrap.cc:654] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.301906 23975 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.302594 23975 ts_tablet_manager.cc:1403] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.009s
I20260709 12:12:45.304229 23976 tablet_bootstrap.cc:654] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.305485 23975 raft_consensus.cc:359] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.306175 23975 raft_consensus.cc:385] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.306473 23975 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.306905 24019 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.307459 24019 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.310787 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:45.312882 23825 leader_election.cc:343] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.315582 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.316027 23976 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.316641 23976 ts_tablet_manager.cc:1403] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.006s
W20260709 12:12:45.317325 23824 leader_election.cc:343] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.317878 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:45.318800 23981 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:45.319801 23983 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.319648 23976 raft_consensus.cc:359] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.320493 23976 raft_consensus.cc:385] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.320379 23983 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.320863 23976 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.321643 23976 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.323982 23976 ts_tablet_manager.cc:1434] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:45.325241 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.326059 23784 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:45.307152 23975 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.327759 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.329114 24002 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.329528 24002 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.329879 24002 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.330015 23975 ts_tablet_manager.cc:1434] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 12:12:45.333325 24019 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.322778 23983 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.345403 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.353545 23860 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:45.345696 23975 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.346424 23976 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.348376 23981 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.358582 23981 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.361301 23981 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.363869 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.364354 23976 tablet_bootstrap.cc:654] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:45.365603 23824 leader_election.cc:343] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.352634 24019 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.366854 24019 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.371701 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.372566 23934 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:45.374064 23975 tablet_bootstrap.cc:654] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.374078 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.375236 23981 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.375968 23981 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.376340 23981 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.379737 24019 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.381150 24002 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.383703 24003 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.391752 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:45.392467 23860 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.402158 23860 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:45.403780 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.405237 23983 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:45.408481 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:45.409212 23860 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.416841 24021 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.417506 24021 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.421723 24021 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.423158 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.423933 23859 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:45.425508 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:45.426676 24021 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.427067 24021 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.427416 24021 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.436709 24021 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.439981 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:45.440678 23859 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.455693 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:45.456755 23784 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.463886 24018 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.464447 24018 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.466595 24018 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.467105 24018 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.469612 24018 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.472738 24018 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.474704 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:45.475052 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.475454 23783 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.475924 23782 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:45.477532 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.480947 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.485466 23783 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:45.493212 23859 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:45.494905 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:45.499831 24057 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:45.501319 24002 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.504264 24003 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.514174 23981 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.516083 23983 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:45.518162 23983 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.519320 24057 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:45.522501 24057 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.519408 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.525126 23933 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:45.520233 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.521449 24097 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.527390 23934 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:45.527227 24097 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.529500 24097 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.531783 23975 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:45.534824 23975 ts_tablet_manager.cc:1403] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.189s	user 0.016s	sys 0.005s
I20260709 12:12:45.537657 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.538470 23858 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:45.540086 23975 raft_consensus.cc:359] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.540915 23975 raft_consensus.cc:385] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.541216 23975 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.541966 23975 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.540444 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.543855 24019 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.544369 24019 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.545516 24019 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.548144 23934 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.556965 23975 ts_tablet_manager.cc:1434] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:12:45.558195 23975 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:45.559114 23751 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:45.560194 24097 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.560564 24097 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.560884 24097 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.578994 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:45.583213 23977 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:45.583767 23977 ts_tablet_manager.cc:1403] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.302s	user 0.118s	sys 0.143s
I20260709 12:12:45.619835 23977 raft_consensus.cc:359] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.620638 23977 raft_consensus.cc:385] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.620929 23977 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.621585 23977 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.628386 24021 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.629765 23784 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:45.634598 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:45.634688 23681 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:45.635942 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.636641 23933 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:45.640373 23981 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.641571 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:45.642180 23933 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.650532 23976 tablet_bootstrap.cc:492] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:45.650648 23977 ts_tablet_manager.cc:1434] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.067s	user 0.014s	sys 0.004s
I20260709 12:12:45.651237 23976 ts_tablet_manager.cc:1403] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.305s	user 0.012s	sys 0.026s
I20260709 12:12:45.652217 23977 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:45.652357 24101 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:45.652879 24101 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.653225 24101 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.639259 24097 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.650856 23860 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:45.661165 24021 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.667610 23977 tablet_bootstrap.cc:654] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.673532 24096 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:12:45.676252 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.667881 24021 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.670136 24057 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:45.668069 24100 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
I20260709 12:12:45.674595 23783 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:12:45.688764 23825 leader_election.cc:343] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.690157 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:45.695839 23682 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:45.680549 24057 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.700747 23976 raft_consensus.cc:359] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.701524 23976 raft_consensus.cc:385] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:45.701800 23976 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.682801 24096 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.708911 23976 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.716470 23976 ts_tablet_manager.cc:1434] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.000s
I20260709 12:12:45.685702 24019 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.707808 23933 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:45.719487 23981 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.713622 24096 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.726389 24021 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.726977 24002 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:12:45.738222 24095 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:45.741333 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:45.771540 23975 tablet_bootstrap.cc:654] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:45.750803 24021 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:12:45.769402 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.775966 24108 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:45.781414 23976 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:45.752660 24095 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:45.744328 24102 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.073s
I20260709 12:12:45.743230 24101 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.777902 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.856108 24101 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.857052 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.819173 24057 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.826460 24018 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.880409 24106 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.881052 24106 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.881314 24103 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 12:12:45.879580 24018 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.882225 23983 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.849694 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:45.936012 24019 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.939208 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.840942 23934 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:45.947791 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:45.938784 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:45.949389 23929 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:45.924317 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:45.950276 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.950646 23983 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.914189 23858 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:45.896152 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.953658 23983 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.912636 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:45.888068 24096 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:45.954833 23932 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:45.954960 23749 leader_election.cc:343] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.898025 24095 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.954879 23857 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:12:45.963302 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 223895us
W20260709 12:12:45.964076 23824 leader_election.cc:343] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:45.965515 23825 leader_election.cc:343] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.974054 23929 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:45.974913 24097 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.934340 24106 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:45.977746 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:45.980288 24018 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:45.960659 23749 leader_election.cc:343] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:45.991981 23856 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:45.999799 23977 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:46.000221 23977 ts_tablet_manager.cc:1403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.348s	user 0.031s	sys 0.049s
I20260709 12:12:46.002478 23977 raft_consensus.cc:359] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.003089 23977 raft_consensus.cc:385] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.003293 23977 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:45.991127 24108 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:12:45.990849 23751 leader_election.cc:343] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.004130 23977 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:45.992492 23932 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:46.006249 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:46.008438 23931 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.014361 23977 ts_tablet_manager.cc:1434] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:12:46.015153 23977 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:46.015465 23975 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:46.015873 23975 ts_tablet_manager.cc:1403] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.458s	user 0.010s	sys 0.013s
I20260709 12:12:46.026057 23975 raft_consensus.cc:359] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.026553 23981 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.026782 23975 raft_consensus.cc:385] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.027078 23975 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.018766 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.031898 23975 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.019160 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:46.027521 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:46.041504 23976 tablet_bootstrap.cc:654] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.042112 23975 ts_tablet_manager.cc:1434] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:12:46.042879 23975 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:46.008914 24108 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.033845 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:46.037226 23981 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:46.048836 23981 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:46.047636 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:12:46.048256 23824 leader_election.cc:343] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.026335 23751 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.051656 23934 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:46.057435 23783 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:46.055886 23785 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:46.064381 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 318792us
W20260709 12:12:46.066638 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 103430us
I20260709 12:12:46.067756 24097 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.087482 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.088243 23784 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:46.089221 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.068828 23981 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.098075 23785 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:46.090894 24093 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:12:46.109251 23977 tablet_bootstrap.cc:654] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.101876 24095 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:12:46.092976 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.123445 23976 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:46.124086 23976 ts_tablet_manager.cc:1403] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.343s	user 0.026s	sys 0.041s
I20260709 12:12:46.104210 23749 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.126412 24018 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:46.126892 24018 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:46.127737 24018 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.128388 23976 raft_consensus.cc:359] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.129132 23976 raft_consensus.cc:385] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.129555 24057 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.129402 23976 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.103184 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.130381 23976 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:46.137799 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52570.5us
I20260709 12:12:46.138829 24021 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:46.139218 24021 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.139571 24021 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.144578 23976 ts_tablet_manager.cc:1434] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:12:46.145247 24098 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:12:46.147888 23976 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
W20260709 12:12:46.149091 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:46.166815 23931 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:46.151196 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.173833 23682 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:46.329254 24098 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.280613 24021 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.207849 24110 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:46.211112 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.339951 24098 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:46.360502 24112 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.176s	user 0.021s	sys 0.012s
W20260709 12:12:46.333271 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 190387us
I20260709 12:12:46.338511 24106 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.404423 23782 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:46.370503 24110 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.407145 24021 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:46.414238 23976 tablet_bootstrap.cc:654] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.414772 24098 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:46.420186 24002 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.420846 24002 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.410742 24110 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:46.409618 23825 leader_election.cc:343] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.420197 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:46.403986 24106 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.443039 24106 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:46.446069 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.449689 23975 tablet_bootstrap.cc:654] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:46.427538 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 82785.4us
I20260709 12:12:46.466214 24002 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:46.447347 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 112004us
I20260709 12:12:46.438870 23825 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.482589 24134 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:12:46.495839 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 89825.8us
I20260709 12:12:46.492226 24106 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.502470 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.500051 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:46.506927 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:46.514803 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:46.514403 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:46.514765 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:46.522330 23928 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:46.521245 23858 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:46.514027 24134 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:12:46.542303 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:46.528033 24093 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.549985 23977 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:46.550781 23977 ts_tablet_manager.cc:1403] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.536s	user 0.015s	sys 0.022s
I20260709 12:12:46.556170 24095 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:12:46.556509 23900 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.562053 24093 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:46.562482 24093 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.562851 24093 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:46.545641 23824 leader_election.cc:343] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.574023 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:46.575018 24095 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:46.575419 23899 leader_election.cc:343] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.590541 23975 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:46.591046 23975 ts_tablet_manager.cc:1403] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.548s	user 0.026s	sys 0.028s
W20260709 12:12:46.590571 24138 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:12:46.596652 23749 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.596815 23681 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:46.594316 24095 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 12:12:46.603010 23749 leader_election.cc:343] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.605113 23785 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.608474 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.588362 24110 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.623425 23860 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:46.623832 24019 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:46.625985 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.626875 24098 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.616015 24093 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.629496 24093 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:46.630259 24093 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:46.633095 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:46.633710 23860 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.640748 23860 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:46.641311 24134 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.642305 23900 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.643251 23983 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:46.643752 23983 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:46.644635 23983 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.649904 24110 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:46.653554 23825 leader_election.cc:343] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.654093 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.655153 24019 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.657357 24106 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:46.658644 24106 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:46.661765 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:46.662621 23783 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:46.665460 23977 raft_consensus.cc:359] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.665886 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:46.666250 23977 raft_consensus.cc:385] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.666559 23977 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.667182 23977 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.673048 23976 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:46.677451 23976 ts_tablet_manager.cc:1403] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.531s	user 0.005s	sys 0.079s
I20260709 12:12:46.687278 23976 raft_consensus.cc:359] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.687875 23976 raft_consensus.cc:385] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.688089 23976 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.688752 23977 ts_tablet_manager.cc:1434] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.138s	user 0.008s	sys 0.000s
W20260709 12:12:46.671777 23824 leader_election.cc:343] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.699118 23976 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.711251 23976 ts_tablet_manager.cc:1434] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:12:46.712134 23976 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:46.712828 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:46.720767 23975 raft_consensus.cc:359] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.721508 23975 raft_consensus.cc:385] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.721792 23975 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.722414 23975 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.726001 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:46.726585 24105 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:46.727566 24110 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:46.728127 23975 ts_tablet_manager.cc:1434] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.137s	user 0.006s	sys 0.001s
I20260709 12:12:46.729063 23975 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:46.735414 23977 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:46.741084 24106 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:46.745693 23899 leader_election.cc:343] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:46.739646 23683 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:46.748039 23683 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:46.769330 23976 tablet_bootstrap.cc:654] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.787642 23975 tablet_bootstrap.cc:654] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.789544 23977 tablet_bootstrap.cc:654] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.823397 23681 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:46.858132 24105 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:46.861248 23976 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:46.864629 23976 ts_tablet_manager.cc:1403] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.153s	user 0.014s	sys 0.000s
I20260709 12:12:46.865871 24096 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.866391 24096 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.871559 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:46.871461 23975 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:46.872607 23860 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:46.873543 23975 ts_tablet_manager.cc:1403] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.145s	user 0.015s	sys 0.007s
I20260709 12:12:46.874075 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.875005 24002 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:46.875365 24002 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.875742 24002 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.880389 24096 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:46.880626 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:46.881340 23785 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:46.881548 23977 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:46.882128 23977 ts_tablet_manager.cc:1403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.147s	user 0.016s	sys 0.021s
I20260709 12:12:46.884860 24002 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.886018 23975 raft_consensus.cc:359] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.886519 23975 raft_consensus.cc:385] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.886736 23975 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.887296 23975 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.889288 23975 ts_tablet_manager.cc:1434] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:12:46.885026 23977 raft_consensus.cc:359] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.890875 23977 raft_consensus.cc:385] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.891556 23977 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.892692 23977 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.890316 23975 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:46.897406 23977 ts_tablet_manager.cc:1434] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:12:46.897229 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:46.899282 23779 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.899399 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:46.898866 23977 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:46.900115 23860 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:46.901284 24002 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:46.900569 23976 raft_consensus.cc:359] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.902108 23976 raft_consensus.cc:385] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.902390 23976 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.903870 23976 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.907781 23860 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:46.908390 23977 tablet_bootstrap.cc:654] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.909418 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:46.915158 23975 tablet_bootstrap.cc:654] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.918433 24096 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:46.919014 24096 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:46.920228 24096 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.929725 23977 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:46.930356 23977 ts_tablet_manager.cc:1403] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 12:12:46.931996 23975 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:46.932511 23975 ts_tablet_manager.cc:1403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.000s
I20260709 12:12:46.933358 23977 raft_consensus.cc:359] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.934206 23977 raft_consensus.cc:385] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.934525 23977 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.935098 23975 raft_consensus.cc:359] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.935997 23975 raft_consensus.cc:385] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:46.935285 23977 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.936311 23975 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:46.938870 23975 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:46.941382 23975 ts_tablet_manager.cc:1434] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:12:46.939739 23977 ts_tablet_manager.cc:1434] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:12:46.942683 23977 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:46.943544 23975 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:46.946403 23976 ts_tablet_manager.cc:1434] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.081s	user 0.004s	sys 0.003s
I20260709 12:12:46.947434 23976 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:46.952675 23977 tablet_bootstrap.cc:654] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.957891 23975 tablet_bootstrap.cc:654] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:46.977416 23779 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:46.978508 23683 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:46.991961 23976 tablet_bootstrap.cc:654] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.011898 24098 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.012490 24098 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.016335 24098 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.017053 23976 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.017519 23976 ts_tablet_manager.cc:1403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.000s
I20260709 12:12:47.018668 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.019152 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.019526 23860 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.020843 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:12:47.021606 23749 leader_election.cc:343] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.022657 23977 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.021662 24098 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.023118 24098 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.023281 23975 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.023562 24098 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.021871 23976 raft_consensus.cc:359] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.024063 23975 ts_tablet_manager.cc:1403] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.081s	user 0.025s	sys 0.034s
I20260709 12:12:47.024088 23976 raft_consensus.cc:385] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.027084 23975 raft_consensus.cc:359] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.023248 23977 ts_tablet_manager.cc:1403] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.081s	user 0.044s	sys 0.026s
I20260709 12:12:47.030545 23977 raft_consensus.cc:359] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.031309 23977 raft_consensus.cc:385] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.031924 23977 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.031675 24098 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.032788 23977 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.033294 23976 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.034022 23976 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.031666 23975 raft_consensus.cc:385] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.035575 23975 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.036083 23976 ts_tablet_manager.cc:1434] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:12:47.036393 23975 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.035136 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.037187 23929 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.038369 23975 ts_tablet_manager.cc:1434] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:12:47.038348 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.038406 23977 ts_tablet_manager.cc:1434] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:12:47.039062 23860 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.039736 23977 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.039831 23976 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.045740 23860 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.046195 23977 tablet_bootstrap.cc:654] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.047175 23975 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.047192 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.047464 24105 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.046356 23976 tablet_bootstrap.cc:654] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.048019 24105 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.050101 24105 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.051746 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.048273 24134 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.052994 24134 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:47.053798 23975 tablet_bootstrap.cc:654] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.053910 24134 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.058935 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:47.062704 23825 leader_election.cc:343] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:47.074949 23824 leader_election.cc:343] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.075479 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.077666 23981 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.098752 24092 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:47.035876 24096 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.103472 24096 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.105679 24096 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.108001 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.108728 23860 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:47.109992 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:47.110800 24002 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.111181 24002 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.111697 24002 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.115716 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.116813 23779 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:47.123904 24002 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.127501 24002 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.133374 23784 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.133682 24105 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.134243 24105 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.134475 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:47.135142 23779 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.137060 24105 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.140959 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:47.142702 23825 leader_election.cc:343] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.146953 23779 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:47.155864 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.161075 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.161803 23856 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.164714 23899 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:47.165980 24096 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.166664 24096 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:47.167516 24096 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:47.176975 23824 leader_election.cc:343] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.177565 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.178557 24105 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.189587 24092 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:47.190820 23929 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.191362 24105 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.191890 24105 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.195433 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.196812 24105 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:47.197227 23825 leader_election.cc:343] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.203249 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:47.188757 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:47.211805 24098 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:47.217855 23824 leader_election.cc:343] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.218348 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.222344 23856 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:47.226282 24105 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.235889 23977 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.236352 23977 ts_tablet_manager.cc:1403] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.197s	user 0.031s	sys 0.061s
I20260709 12:12:47.238850 23977 raft_consensus.cc:359] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.238749 23683 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.240757 23976 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.241245 23976 ts_tablet_manager.cc:1403] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.202s	user 0.055s	sys 0.090s
I20260709 12:12:47.241809 23975 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.242249 23975 ts_tablet_manager.cc:1403] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.195s	user 0.025s	sys 0.066s
I20260709 12:12:47.244048 23976 raft_consensus.cc:359] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.244567 23976 raft_consensus.cc:385] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.239567 23977 raft_consensus.cc:385] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.244647 23975 raft_consensus.cc:359] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.244966 23977 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.244834 23976 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.245378 23975 raft_consensus.cc:385] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.245684 23975 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.245777 23977 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.246361 23975 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.246068 23976 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.248095 23977 ts_tablet_manager.cc:1434] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:12:47.248639 23975 ts_tablet_manager.cc:1434] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:12:47.249353 23976 ts_tablet_manager.cc:1434] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:12:47.249572 23975 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.250216 23976 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.244817 23681 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:47.256328 23975 tablet_bootstrap.cc:654] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.256708 23976 tablet_bootstrap.cc:654] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.257386 23977 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.264701 23977 tablet_bootstrap.cc:654] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.268069 23975 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.268541 23975 ts_tablet_manager.cc:1403] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:12:47.271440 23975 raft_consensus.cc:359] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.272351 23975 raft_consensus.cc:385] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.272637 23975 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.273303 23975 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.275028 23975 ts_tablet_manager.cc:1434] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:12:47.276335 23975 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.284643 23975 tablet_bootstrap.cc:654] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.296943 23975 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.297519 24134 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.298015 24134 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.300076 24134 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.301857 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.302821 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.303493 23856 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
W20260709 12:12:47.304867 23749 leader_election.cc:343] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.307515 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.308687 24134 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.309058 24134 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.309386 24134 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.321430 24134 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.324378 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.325006 23856 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.331681 23975 ts_tablet_manager.cc:1403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.005s
I20260709 12:12:47.334383 23975 raft_consensus.cc:359] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.335078 23975 raft_consensus.cc:385] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.338982 24096 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:47.347762 23856 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.349050 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.349844 24098 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.350322 24098 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:47.351148 24098 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.335345 23975 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.357739 23975 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.361605 23975 ts_tablet_manager.cc:1434] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.001s
I20260709 12:12:47.362605 23975 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.365839 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:47.367185 23749 leader_election.cc:343] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.369536 24134 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.370770 23983 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:47.377923 23975 tablet_bootstrap.cc:654] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.384435 23976 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.385012 23976 ts_tablet_manager.cc:1403] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.135s	user 0.061s	sys 0.031s
I20260709 12:12:47.388046 23976 raft_consensus.cc:359] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.388840 23976 raft_consensus.cc:385] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.389148 23976 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.389987 23976 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.390515 23977 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.392663 23976 ts_tablet_manager.cc:1434] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:47.393688 23976 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.392738 23977 ts_tablet_manager.cc:1403] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.136s	user 0.036s	sys 0.059s
I20260709 12:12:47.399279 23977 raft_consensus.cc:359] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.400050 23977 raft_consensus.cc:385] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.400333 23977 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.399469 23681 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.400889 23976 tablet_bootstrap.cc:654] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.403543 23977 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.405947 23977 ts_tablet_manager.cc:1434] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.002s
I20260709 12:12:47.406940 23977 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.412420 23976 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.412689 23977 tablet_bootstrap.cc:654] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.412894 23976 ts_tablet_manager.cc:1403] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.006s
I20260709 12:12:47.415742 23976 raft_consensus.cc:359] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.416465 23976 raft_consensus.cc:385] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.416723 23976 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.417497 23976 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.418329 23975 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.418805 23975 ts_tablet_manager.cc:1403] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.002s
I20260709 12:12:47.419972 23976 ts_tablet_manager.cc:1434] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:47.421010 23976 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.421286 23975 raft_consensus.cc:359] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.422068 23975 raft_consensus.cc:385] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.422343 23975 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.422964 23975 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.425228 23975 ts_tablet_manager.cc:1434] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:47.426573 23975 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.428457 23976 tablet_bootstrap.cc:654] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.432514 23977 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.433022 23977 ts_tablet_manager.cc:1403] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 12:12:47.435832 23977 raft_consensus.cc:359] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.436612 23977 raft_consensus.cc:385] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.436874 23977 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.437546 23977 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.440089 23977 ts_tablet_manager.cc:1434] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:47.440281 23975 tablet_bootstrap.cc:654] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.441933 23977 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.443019 24105 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.443547 24105 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.445822 24105 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.447661 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:47.449033 23824 leader_election.cc:343] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.450240 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.451728 23976 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
W20260709 12:12:47.451719 23825 leader_election.cc:343] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.452275 23976 ts_tablet_manager.cc:1403] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.031s	user 0.026s	sys 0.002s
I20260709 12:12:47.452476 23825 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.454516 23981 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.454873 23977 tablet_bootstrap.cc:654] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.455330 23976 raft_consensus.cc:359] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.456069 23976 raft_consensus.cc:385] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.456382 23976 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.457192 23976 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.459244 24134 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.459434 23976 ts_tablet_manager.cc:1434] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:47.459779 24134 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.460139 24019 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.460757 23976 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.460712 24019 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.461892 24134 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.463037 24019 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.467767 23976 tablet_bootstrap.cc:654] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.477589 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.478464 23856 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.479745 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.480187 23975 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.480840 23975 ts_tablet_manager.cc:1403] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.004s
I20260709 12:12:47.480752 24098 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:47.482470 24098 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.482849 24098 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.483156 24098 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.484016 23975 raft_consensus.cc:359] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.484771 23975 raft_consensus.cc:385] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.485038 23975 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.485702 23975 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.488981 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.498281 23975 ts_tablet_manager.cc:1434] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:12:47.499858 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.500813 23929 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.505069 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.505725 23784 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:47.510854 23975 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:12:47.514773 23824 leader_election.cc:343] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.520227 23975 tablet_bootstrap.cc:654] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.520116 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.522513 24019 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.523016 24019 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.523728 24019 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.532397 24137 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:47.533238 23975 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.533782 23975 ts_tablet_manager.cc:1403] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.009s
I20260709 12:12:47.536684 23975 raft_consensus.cc:359] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.537459 23975 raft_consensus.cc:385] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.537739 23975 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.538468 23975 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.552309 23975 ts_tablet_manager.cc:1434] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.004s
I20260709 12:12:47.561592 24098 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.577554 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.578258 23856 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.565608 23975 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.586462 23856 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.572309 24019 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.588403 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.589488 24137 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.589988 24137 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:47.590099 23975 tablet_bootstrap.cc:654] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.590941 24137 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.594131 24134 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.594626 24134 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.597396 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:47.598284 23785 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.603469 23983 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.604054 23983 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.607229 23785 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:47.609297 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.610188 23858 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:47.612231 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.607344 23983 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.613216 24105 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.613742 24105 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:47.614758 24105 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.621526 24134 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.623423 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.624487 23785 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:47.626638 24019 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.629681 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:47.632323 23749 leader_election.cc:343] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.627024 23899 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:47.634075 23983 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.635687 23976 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.636406 23976 ts_tablet_manager.cc:1403] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.176s	user 0.064s	sys 0.078s
I20260709 12:12:47.637301 23977 tablet_bootstrap.cc:492] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.637840 23977 ts_tablet_manager.cc:1403] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.196s	user 0.062s	sys 0.068s
I20260709 12:12:47.639967 24093 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:47.639967 23976 raft_consensus.cc:359] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.641777 23976 raft_consensus.cc:385] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.642043 23976 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.642460 23983 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.642683 23976 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.644335 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.644973 23976 ts_tablet_manager.cc:1434] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 12:12:47.645649 23824 leader_election.cc:343] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.645972 23976 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.641151 23977 raft_consensus.cc:359] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.649523 23977 raft_consensus.cc:385] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.649806 23977 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.632524 23975 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.650810 23975 ts_tablet_manager.cc:1403] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.085s	user 0.010s	sys 0.003s
I20260709 12:12:47.650537 23977 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.652534 23976 tablet_bootstrap.cc:654] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.653015 23977 ts_tablet_manager.cc:1434] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:12:47.653949 23977 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.653755 23975 raft_consensus.cc:359] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.654487 23975 raft_consensus.cc:385] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.654788 23975 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.656167 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.656886 23856 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.658044 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.659442 24134 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.659905 24134 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.662547 24134 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.666617 24093 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.667121 24093 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.667699 23977 tablet_bootstrap.cc:654] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.669898 24134 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.670161 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.671110 23785 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:47.672839 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.673465 23855 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.673664 24093 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.675005 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.676894 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:47.678187 24093 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.681102 23855 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.682428 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.685621 24093 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.687435 24196 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.689050 24134 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.690420 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.703023 23681 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.709400 23975 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.710721 24196 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:47.724869 24196 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.711301 24098 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.716064 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.748724 23975 ts_tablet_manager.cc:1434] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.000s
I20260709 12:12:47.753922 23975 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:47.748132 23934 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.748991 24137 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.771822 23976 tablet_bootstrap.cc:492] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:47.772331 23976 ts_tablet_manager.cc:1403] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.039s
I20260709 12:12:47.765189 23929 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.763023 24137 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.779989 23977 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.782035 23977 ts_tablet_manager.cc:1403] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.128s	user 0.026s	sys 0.020s
I20260709 12:12:47.786167 24092 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:47.788584 23976 raft_consensus.cc:359] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.789475 23976 raft_consensus.cc:385] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.789739 23976 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.790441 23976 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.793576 23976 ts_tablet_manager.cc:1434] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:12:47.794452 23976 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:47.797819 23977 raft_consensus.cc:359] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.798553 23977 raft_consensus.cc:385] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:47.798826 23977 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:47.799556 23977 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.801358 23682 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.803356 23977 ts_tablet_manager.cc:1434] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:12:47.804371 23977 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:47.812467 24092 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.819660 24092 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.824749 24137 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.842054 23934 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.848549 24096 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.849117 24096 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.854067 24137 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.854693 24137 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.871645 24096 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.872313 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.872987 23860 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.874166 24196 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.879727 24137 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.886488 24196 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.882580 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.893913 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.894635 23860 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.895386 24092 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:47.897884 23975 tablet_bootstrap.cc:654] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.900075 23929 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:47.900146 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.901234 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.904244 24137 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.904633 24137 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.904999 24137 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.894833 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:47.906558 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.907071 23785 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:47.907265 23860 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:47.909492 23976 tablet_bootstrap.cc:654] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.909744 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.912562 23931 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.910772 23900 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:47.913718 24092 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.914047 24092 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.914309 24092 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.924853 24196 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.923730 23681 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:47.926056 23749 leader_election.cc:343] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.929330 24098 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:47.922477 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.934425 23858 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:47.938391 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:47.940441 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.941319 24098 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:47.941685 24098 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.942047 24098 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.946065 24137 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.948314 24137 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.949782 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:47.950654 24098 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.950954 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.951511 23856 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:47.953864 24098 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:47.955788 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.956928 23977 tablet_bootstrap.cc:654] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:47.957605 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:47.959874 23929 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:47.963269 23751 leader_election.cc:343] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.965509 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:47.966106 23858 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:47.970053 23749 leader_election.cc:343] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.969594 23856 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.970865 23975 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:47.971622 23975 ts_tablet_manager.cc:1403] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.218s	user 0.022s	sys 0.008s
W20260709 12:12:47.970856 23749 leader_election.cc:343] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:47.976590 23749 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:47.976787 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:47.977391 24137 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.977481 24196 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:47.978072 24196 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:47.978940 24196 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.979406 23858 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:47.960574 23900 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:47.982100 23977 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:47.982573 23977 ts_tablet_manager.cc:1403] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.007s
I20260709 12:12:47.983915 24096 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:47.997607 24092 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.018272 23976 tablet_bootstrap.cc:492] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
W20260709 12:12:48.018469 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 60358.5us
I20260709 12:12:48.018788 23976 ts_tablet_manager.cc:1403] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.225s	user 0.009s	sys 0.004s
I20260709 12:12:48.021335 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.021318 24098 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.021624 24198 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.022303 24137 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.022768 24137 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:48.023607 24137 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:47.986483 23929 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:48.034441 23976 raft_consensus.cc:359] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.035147 23976 raft_consensus.cc:385] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.035390 23976 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.036077 23976 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.038131 23976 ts_tablet_manager.cc:1434] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:12:48.039043 23976 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.040426 23975 raft_consensus.cc:359] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.041049 23975 raft_consensus.cc:385] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.041283 23975 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.041854 23975 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.043759 23975 ts_tablet_manager.cc:1434] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.072s	user 0.010s	sys 0.014s
I20260709 12:12:48.044674 23975 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.048751 24106 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.049193 23976 tablet_bootstrap.cc:654] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.049256 24106 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.049487 23977 raft_consensus.cc:359] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.050062 23975 tablet_bootstrap.cc:654] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.050459 23977 raft_consensus.cc:385] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.052910 23977 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.054221 23977 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.070638 23977 ts_tablet_manager.cc:1434] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.004s
I20260709 12:12:48.059453 23975 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.061050 23976 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.080627 23975 ts_tablet_manager.cc:1403] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.036s	user 0.006s	sys 0.008s
I20260709 12:12:48.080730 23976 ts_tablet_manager.cc:1403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.003s
I20260709 12:12:48.060987 24106 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.065807 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.066030 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:48.083263 23858 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:48.083971 23824 leader_election.cc:343] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.085831 23976 raft_consensus.cc:359] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.086504 23976 raft_consensus.cc:385] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.086773 23976 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.083451 23975 raft_consensus.cc:359] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.087463 23975 raft_consensus.cc:385] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.055874 24093 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.087965 23975 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.073907 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.088048 24093 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.088617 23779 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.088953 23975 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.090199 24093 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.074676 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.090859 23781 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:48.091387 23975 ts_tablet_manager.cc:1434] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:12:48.091681 23929 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.092451 23975 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.066859 24092 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.079324 23681 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.095150 23977 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.087423 23976 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.103554 23825 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.108758 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.112807 24106 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.113221 24106 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.113559 24106 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.110321 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.099561 23779 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:48.098738 23858 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:48.111241 23975 tablet_bootstrap.cc:654] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.116384 23781 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:48.097417 24098 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:12:48.118602 23976 ts_tablet_manager.cc:1434] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.038s	user 0.002s	sys 0.005s
I20260709 12:12:48.119627 23976 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.122285 23977 tablet_bootstrap.cc:654] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.125456 23899 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:48.126350 24092 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.127036 24092 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:48.127527 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.127898 24092 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.133373 24105 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.136229 24106 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.138396 24093 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.138856 24106 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.140031 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:48.141767 23824 leader_election.cc:343] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.142649 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.143285 23779 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.145879 24105 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.160671 23779 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.161402 23682 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:48.173573 24137 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.175510 23825 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.176388 24106 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.179327 23977 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.179975 23977 ts_tablet_manager.cc:1403] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.085s	user 0.018s	sys 0.014s
I20260709 12:12:48.179456 23976 tablet_bootstrap.cc:654] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.176908 24106 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:48.181792 24106 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.182930 23977 raft_consensus.cc:359] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.183774 23977 raft_consensus.cc:385] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.184083 23977 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.185235 23977 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.187348 23977 ts_tablet_manager.cc:1434] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:48.188373 23977 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.194411 23977 tablet_bootstrap.cc:654] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.201983 23975 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.202495 23975 ts_tablet_manager.cc:1403] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.019s
I20260709 12:12:48.205349 23975 raft_consensus.cc:359] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.205224 23680 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.219885 24137 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.236310 23976 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.235716 23682 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.236871 23976 ts_tablet_manager.cc:1403] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.117s	user 0.010s	sys 0.000s
I20260709 12:12:48.240064 23976 raft_consensus.cc:359] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.240820 23976 raft_consensus.cc:385] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.241094 23976 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.241827 23976 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.244148 23976 ts_tablet_manager.cc:1434] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:48.206172 23975 raft_consensus.cc:385] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.208601 23977 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.245204 23976 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.245553 23977 ts_tablet_manager.cc:1403] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.001s
I20260709 12:12:48.247047 24092 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.247537 24092 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.248301 23977 raft_consensus.cc:359] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.248943 23977 raft_consensus.cc:385] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.249203 23977 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.249644 24092 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.249913 23977 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.252101 23977 ts_tablet_manager.cc:1434] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:48.253014 23977 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.253489 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.254354 23856 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.255163 23976 tablet_bootstrap.cc:654] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.255676 23975 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.256428 23975 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.269397 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.275787 24002 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.276105 24199 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.276610 24199 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.278787 24199 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.281409 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.282835 24096 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.283842 23977 tablet_bootstrap.cc:654] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.287344 23975 ts_tablet_manager.cc:1434] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.002s
I20260709 12:12:48.287950 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.288440 24002 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.288400 23975 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:12:48.290362 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:48.290642 24002 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:48.292670 23749 leader_election.cc:343] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.292992 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.293656 23781 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:48.295033 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.295704 23858 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:48.296149 23899 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:48.295032 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 93379.3us
I20260709 12:12:48.297221 24093 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.297619 24093 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.297992 24093 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.301687 23975 tablet_bootstrap.cc:654] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.305321 24096 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.307207 23977 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.309194 23976 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.310822 23976 ts_tablet_manager.cc:1403] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.015s
I20260709 12:12:48.312395 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.313072 23856 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:48.313737 23976 raft_consensus.cc:359] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.314473 23976 raft_consensus.cc:385] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.314599 24105 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.314762 23976 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.315078 24105 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.315932 23977 ts_tablet_manager.cc:1403] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.001s
I20260709 12:12:48.317124 24105 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.318802 23977 raft_consensus.cc:359] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.319904 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.321043 23976 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.321755 24201 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.322225 24201 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.322613 24201 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.325145 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.330085 24201 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.331477 23779 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.332499 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.333264 24093 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.335966 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.336632 23779 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.336510 23976 ts_tablet_manager.cc:1434] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:12:48.340643 23976 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.343698 24201 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.319556 23977 raft_consensus.cc:385] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.344619 23977 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.345412 23977 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.345918 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:48.346534 23860 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.347784 23977 ts_tablet_manager.cc:1434] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:12:48.349165 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:48.350678 23749 leader_election.cc:343] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.352751 23977 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.353461 23975 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.353914 23975 ts_tablet_manager.cc:1403] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.066s	user 0.023s	sys 0.027s
I20260709 12:12:48.354087 24093 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.355455 23779 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:48.353530 23860 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:48.356606 23975 raft_consensus.cc:359] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.357326 23975 raft_consensus.cc:385] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.351469 23976 tablet_bootstrap.cc:654] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.357468 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:48.358641 24096 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.359197 24096 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:48.359674 23977 tablet_bootstrap.cc:654] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.359961 23975 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.360662 23975 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.363240 23975 ts_tablet_manager.cc:1434] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:12:48.365053 23975 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.361423 24096 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.375800 23975 tablet_bootstrap.cc:654] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.377947 23977 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.378965 23977 ts_tablet_manager.cc:1403] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.001s
I20260709 12:12:48.381989 23977 raft_consensus.cc:359] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.383035 23977 raft_consensus.cc:385] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.383348 23977 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.384104 23977 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.387456 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:48.388078 23860 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.388398 23977 ts_tablet_manager.cc:1434] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:12:48.389443 23977 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.389535 24227 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.390023 24227 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.393666 23975 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.394177 23975 ts_tablet_manager.cc:1403] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.006s
I20260709 12:12:48.394258 23860 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:48.395015 24019 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:48.387393 23824 leader_election.cc:343] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.395911 23977 tablet_bootstrap.cc:654] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.397380 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.399672 24105 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.402100 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.395489 24019 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.396745 23975 raft_consensus.cc:359] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.402899 24201 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.403081 23975 raft_consensus.cc:385] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.403401 23975 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.403425 24201 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:48.404251 23975 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.404378 24201 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.405277 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.405906 23779 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:48.406415 23975 ts_tablet_manager.cc:1434] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:12:48.406288 24199 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.407434 24227 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.408926 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.409278 24199 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.407347 23975 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.409983 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.410615 23779 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.411854 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.412266 24225 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.412636 24225 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.412885 24105 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:48.412976 24225 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.413326 24105 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.413668 24105 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:48.409732 24199 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.414849 23680 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:48.417150 24199 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.418736 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:48.420162 23749 leader_election.cc:343] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.416262 23975 tablet_bootstrap.cc:654] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.423185 23977 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.423895 23977 ts_tablet_manager.cc:1403] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.001s
I20260709 12:12:48.426858 23977 raft_consensus.cc:359] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.427687 23977 raft_consensus.cc:385] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.427984 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.428004 23977 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.429589 24105 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.430128 23977 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.431905 23977 ts_tablet_manager.cc:1434] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:12:48.432473 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.433060 23779 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:48.436533 24225 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.438966 24225 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.439702 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.440584 23781 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.440834 24019 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.441936 23976 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.442405 23976 ts_tablet_manager.cc:1403] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.018s
I20260709 12:12:48.442440 23779 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:48.445335 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:48.445488 23976 raft_consensus.cc:359] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.446171 23976 raft_consensus.cc:385] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.446038 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.446532 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.446481 23976 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.446713 23929 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.450367 23976 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.452732 24105 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.454550 23977 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.459885 24201 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.460356 23976 ts_tablet_manager.cc:1434] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 12:12:48.462211 23976 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.463497 23781 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.462972 23681 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.470155 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.471154 24106 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.471648 24106 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:12:48.472512 23824 leader_election.cc:343] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.472545 24106 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:48.473533 23824 leader_election.cc:343] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.475092 23977 tablet_bootstrap.cc:654] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.476094 24019 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:48.481232 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.494351 23976 tablet_bootstrap.cc:654] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.496665 24019 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:48.497617 24019 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.500075 24224 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.500640 24224 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.501855 23860 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:48.502830 24224 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.504312 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.505053 23781 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:48.506664 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.507513 24224 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.507926 24224 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.508258 24224 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.509192 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:48.509807 23929 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:48.512988 23976 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.515074 23976 ts_tablet_manager.cc:1403] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.012s
I20260709 12:12:48.514518 23977 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.518556 23977 ts_tablet_manager.cc:1403] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.014s
I20260709 12:12:48.518088 23976 raft_consensus.cc:359] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.518977 23976 raft_consensus.cc:385] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.519330 23976 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.520255 23975 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.520159 23976 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.520750 23975 ts_tablet_manager.cc:1403] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.114s	user 0.024s	sys 0.017s
I20260709 12:12:48.521286 24002 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.521116 23977 raft_consensus.cc:359] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.521788 23977 raft_consensus.cc:385] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.522133 23977 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.521749 24002 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.522749 23977 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.523898 24002 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.524847 23977 ts_tablet_manager.cc:1434] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:12:48.525277 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.525769 23977 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.526156 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.526909 24201 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.528726 24201 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.529080 24201 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.529425 24201 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.531549 24225 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.532418 23977 tablet_bootstrap.cc:654] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.532112 24225 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.534214 24225 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.536442 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.536597 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:48.537194 23824 leader_election.cc:343] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.538367 23929 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:48.539577 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.540359 23975 raft_consensus.cc:359] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.541019 23975 raft_consensus.cc:385] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.541172 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.541932 23779 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:12:48.546919 23824 leader_election.cc:343] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.541303 23975 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.545284 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.551295 24096 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.551728 23975 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.558145 23781 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:48.560988 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.562137 24105 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.562551 24105 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.562908 24105 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.564831 23975 ts_tablet_manager.cc:1434] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:12:48.564548 24199 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.565845 23975 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.570320 24105 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.571887 24224 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.574250 23975 tablet_bootstrap.cc:654] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.574502 24105 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.586493 24201 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.589102 24201 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.589340 24093 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.590899 23976 ts_tablet_manager.cc:1434] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
I20260709 12:12:48.591419 24096 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.601338 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:48.612237 23749 leader_election.cc:343] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.594110 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:48.619480 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:48.621062 23856 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:48.614353 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.622570 23781 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.623895 24019 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:48.630431 23824 leader_election.cc:343] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.630523 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.631660 24199 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.636174 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.637055 23779 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.637565 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:48.639062 23824 leader_election.cc:343] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.639497 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.639954 23975 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.639505 24224 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.640441 23975 ts_tablet_manager.cc:1403] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.002s
I20260709 12:12:48.641319 24224 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.641099 23781 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.642755 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.643126 23975 raft_consensus.cc:359] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.643759 24106 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.644295 24106 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:48.645015 23976 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.645167 24106 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.643846 23975 raft_consensus.cc:385] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.647497 23975 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.648254 23977 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.648789 23977 ts_tablet_manager.cc:1403] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.123s	user 0.046s	sys 0.033s
I20260709 12:12:48.650331 23975 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.651043 24019 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.651621 24019 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.653142 23975 ts_tablet_manager.cc:1434] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:12:48.654270 23975 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.654778 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.654927 24019 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.655449 23779 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:48.656970 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:48.658043 23976 tablet_bootstrap.cc:654] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.659541 23977 raft_consensus.cc:359] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.660319 23977 raft_consensus.cc:385] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.660589 23977 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.661296 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.662489 24224 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:12:48.662698 23824 leader_election.cc:343] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.664268 23975 tablet_bootstrap.cc:654] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.672479 24105 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.673954 24199 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.674541 24199 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.679709 24224 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.676512 23977 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.680245 24224 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.684930 23977 ts_tablet_manager.cc:1434] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.004s
I20260709 12:12:48.685985 23977 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.687201 24096 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.688261 24096 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.691895 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.692135 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.693035 23781 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:12:48.694698 23749 leader_election.cc:343] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.695565 24096 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.697876 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.700276 24093 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.706028 23681 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.710033 23681 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 2, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.708861 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.686013 24199 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.722748 24096 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.715632 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.731561 24224 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.735782 24224 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.729028 24019 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.739012 24096 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.745797 24019 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.750663 23856 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:48.753099 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.754299 24199 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:48.754669 24199 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.755009 24199 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.754150 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:48.762584 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:48.764664 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.770963 23785 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:48.773085 23824 leader_election.cc:343] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.774192 23977 tablet_bootstrap.cc:654] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.778648 24199 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.779881 23779 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.783210 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:48.783915 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.784386 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:12:48.784883 23749 leader_election.cc:343] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:48.785512 23824 leader_election.cc:343] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.786041 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.784955 23856 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.787563 23975 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.777022 24019 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.796197 23975 ts_tablet_manager.cc:1403] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.142s	user 0.027s	sys 0.026s
I20260709 12:12:48.796655 23856 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:48.797220 23976 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.797279 24224 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.797835 23976 ts_tablet_manager.cc:1403] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.153s	user 0.030s	sys 0.020s
I20260709 12:12:48.799508 23975 raft_consensus.cc:359] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.800833 23975 raft_consensus.cc:385] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.801609 23975 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.801404 23976 raft_consensus.cc:359] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.800261 23785 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:48.802820 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.806912 24137 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.805898 23976 raft_consensus.cc:385] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.807444 24137 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:48.807479 23976 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.804936 23975 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.808427 24137 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.808594 23976 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.810245 23976 ts_tablet_manager.cc:1434] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:12:48.811540 23976 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.812597 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.813746 24224 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.814033 24002 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.814229 24224 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:48.814658 24002 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.817819 23975 ts_tablet_manager.cc:1434] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:12:48.818825 23975 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.818848 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.820425 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.820739 24002 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:48.821128 23860 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.822780 23900 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.815099 24224 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.823781 24096 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.826685 23975 tablet_bootstrap.cc:654] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.828449 23976 tablet_bootstrap.cc:654] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.836158 24093 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:48.838790 23976 tablet_bootstrap.cc:492] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.839284 23976 ts_tablet_manager.cc:1403] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.028s	user 0.005s	sys 0.005s
I20260709 12:12:48.786041 24199 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.849275 23976 raft_consensus.cc:359] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.850062 23976 raft_consensus.cc:385] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.850327 23976 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.850425 23680 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.851795 24092 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.854069 23976 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.859658 23976 ts_tablet_manager.cc:1434] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:12:48.861099 23976 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.835505 24096 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.881151 23975 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.881824 23975 ts_tablet_manager.cc:1403] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.011s
I20260709 12:12:48.893218 24196 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.893780 24196 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.897831 23975 raft_consensus.cc:359] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.898578 23975 raft_consensus.cc:385] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.898483 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:48.898945 23975 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
W20260709 12:12:48.899931 23749 leader_election.cc:343] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.899713 23975 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.901901 23975 ts_tablet_manager.cc:1434] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:12:48.902045 24196 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.902884 23975 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.903535 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:48.905263 23751 leader_election.cc:343] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:48.905670 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:48.906754 24199 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:48.912492 23975 tablet_bootstrap.cc:654] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.913538 23976 tablet_bootstrap.cc:654] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.915723 23977 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.916177 23977 ts_tablet_manager.cc:1403] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.230s	user 0.010s	sys 0.037s
I20260709 12:12:48.918624 23977 raft_consensus.cc:359] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.919271 23977 raft_consensus.cc:385] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.919503 23977 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.920120 23977 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.922358 23977 ts_tablet_manager.cc:1434] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 12:12:48.923763 23977 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.924896 23975 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.925437 23975 ts_tablet_manager.cc:1403] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.002s
I20260709 12:12:48.928393 23975 raft_consensus.cc:359] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.929174 23975 raft_consensus.cc:385] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.929450 23975 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.931478 23975 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.932698 23976 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.933168 23976 ts_tablet_manager.cc:1403] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.017s
I20260709 12:12:48.934330 23683 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.935696 23976 raft_consensus.cc:359] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.936365 23976 raft_consensus.cc:385] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.936074 23683 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:48.936650 23976 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.937299 23976 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.938565 23975 ts_tablet_manager.cc:1434] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:12:48.939253 23976 ts_tablet_manager.cc:1434] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:48.939509 23975 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:48.940114 23976 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:48.940905 23977 tablet_bootstrap.cc:654] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.946897 23976 tablet_bootstrap.cc:654] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.950637 24199 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:48.951828 23975 tablet_bootstrap.cc:654] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.953004 24002 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.953485 24002 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.956394 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:48.957157 23785 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:48.958281 23976 tablet_bootstrap.cc:492] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:48.956514 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:48.958770 23976 ts_tablet_manager.cc:1403] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.002s
I20260709 12:12:48.959046 23860 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:48.960237 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:48.960999 24092 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.961587 23976 raft_consensus.cc:359] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.962321 23976 raft_consensus.cc:385] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.962587 23976 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.969591 23977 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:48.970085 23977 ts_tablet_manager.cc:1403] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.002s
I20260709 12:12:48.972951 23977 raft_consensus.cc:359] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.973603 23977 raft_consensus.cc:385] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.973865 23977 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.974452 23977 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.963559 23976 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.982191 23977 ts_tablet_manager.cc:1434] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:12:48.983160 23977 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:48.983882 23975 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:48.984339 23975 ts_tablet_manager.cc:1403] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.015s
I20260709 12:12:48.987423 23975 raft_consensus.cc:359] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:48.988157 23975 raft_consensus.cc:385] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.988415 23975 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.989584 23976 ts_tablet_manager.cc:1434] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 12:12:48.994446 24227 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.989086 23975 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.001858 24227 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.003221 23975 ts_tablet_manager.cc:1434] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.008s
I20260709 12:12:49.006012 23975 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:49.005986 24227 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:48.989044 23977 tablet_bootstrap.cc:654] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.008531 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:49.009229 23781 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:49.010308 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:49.011462 24019 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:49.011956 24019 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.012264 24019 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:49.016420 23975 tablet_bootstrap.cc:654] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.026343 24019 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.043466 24106 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.043663 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:49.044338 23785 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:49.044171 24106 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.046254 24106 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:49.047976 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:49.048646 23781 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:49.028157 23975 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:49.031881 24002 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:49.049623 23975 ts_tablet_manager.cc:1403] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.044s	user 0.010s	sys 0.004s
I20260709 12:12:49.051182 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:49.051853 23785 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:49.052738 24019 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:49.053301 24019 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:49.053169 23975 raft_consensus.cc:359] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.053673 24019 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.053895 23975 raft_consensus.cc:385] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.054013 24019 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:49.054214 23975 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.054982 23975 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.056159 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:49.058036 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:49.058939 24105 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:49.059406 24105 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:49.060290 24105 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.061273 23975 ts_tablet_manager.cc:1434] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 12:12:49.062251 23975 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:49.069156 23975 tablet_bootstrap.cc:654] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.034994 24137 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:49.087498 23824 leader_election.cc:343] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:49.035877 24092 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:49.035799 23976 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:49.036751 24224 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:49.094623 24096 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.109275 23977 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:49.125473 23977 ts_tablet_manager.cc:1403] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.143s	user 0.020s	sys 0.016s
I20260709 12:12:49.096712 24092 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.109577 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:49.136207 24092 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.135514 24096 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.240963 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:49.250160 23976 tablet_bootstrap.cc:654] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.253036 23929 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:49.256629 24019 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.258730 23975 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:49.263494 23975 ts_tablet_manager.cc:1403] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.201s	user 0.039s	sys 0.019s
I20260709 12:12:49.266279 24019 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:49.266655 23976 tablet_bootstrap.cc:492] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:49.277287 23976 ts_tablet_manager.cc:1403] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.242s	user 0.011s	sys 0.002s
I20260709 12:12:49.280717 23976 raft_consensus.cc:359] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.281464 23976 raft_consensus.cc:385] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.281752 23976 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.282431 23976 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.273480 23977 raft_consensus.cc:359] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.284632 23977 raft_consensus.cc:385] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.284981 23977 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.285976 23976 ts_tablet_manager.cc:1434] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:12:49.285637 23977 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.286888 23976 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:49.287775 23977 ts_tablet_manager.cc:1434] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.162s	user 0.006s	sys 0.004s
I20260709 12:12:49.288765 23977 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:49.268646 23975 raft_consensus.cc:359] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.293236 23975 raft_consensus.cc:385] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.293574 23975 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.294204 23976 tablet_bootstrap.cc:654] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.294246 23975 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.297410 23975 ts_tablet_manager.cc:1434] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:12:49.298662 23975 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:49.299978 23976 tablet_bootstrap.cc:492] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:49.300465 23976 ts_tablet_manager.cc:1403] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:12:49.301265 23977 tablet_bootstrap.cc:654] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.303342 23976 raft_consensus.cc:359] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.304104 23976 raft_consensus.cc:385] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.304378 23976 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.305058 23976 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.307128 23976 ts_tablet_manager.cc:1434] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:49.308142 23976 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:49.315203 24096 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:49.315755 23975 tablet_bootstrap.cc:654] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.317152 24092 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:49.321288 23976 tablet_bootstrap.cc:654] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.327556 23977 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:49.330276 23977 ts_tablet_manager.cc:1403] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.006s
I20260709 12:12:49.333343 23975 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:49.333855 23975 ts_tablet_manager.cc:1403] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.003s
I20260709 12:12:49.333511 23977 raft_consensus.cc:359] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:49.334383 23977 raft_consensus.cc:385] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.334713 23977 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
W20260709 12:12:50.420091 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800164bc0 after lost signal to thread 22632
W20260709 12:12:50.513561 22634 debug-util.cc:398] Leaking SignalData structure 0x7b080000f960 after lost signal to thread 23826
W20260709 12:12:50.514173 22634 kernel_stack_watchdog.cc:198] Thread 23826 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 415ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:12:49.335448 23977 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.151073 23977 ts_tablet_manager.cc:1434] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 1.820s	user 0.014s	sys 0.012s
W20260709 12:12:51.151444 23977 ts_tablet_manager.cc:1467] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Tablet startup took 1820ms
W20260709 12:12:51.151786 23977 ts_tablet_manager.cc:1469] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Trace:
0709 12:12:43.851543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.851757 (+   214us) service_pool.cc:224] Handling call
0709 12:12:43.872007 (+ 20250us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:43.872116 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:43.884795 (+ 12679us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:43.907932 (+ 23137us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:43.954385 (+ 46453us) inbound_call.cc:177] Queueing success response
0709 12:12:49.288444 (+5334059us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:12:49.330615 (+ 42171us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:12:49.331675 (+  1060us) tablet_replica.cc:261] Starting instrumentation
0709 12:12:49.332178 (+   503us) tablet_replica.cc:280] Starting consensus
Metrics: {"tablet-open.queue_time_us":5340608}
I20260709 12:12:51.154105 23977 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:51.160835 23977 tablet_bootstrap.cc:654] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.165542 23977 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:51.165567 24286 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.165967 23977 ts_tablet_manager.cc:1403] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:12:51.166160 24286 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.168438 24286 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.168478 23977 raft_consensus.cc:359] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.169150 24286 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.169399 23977 raft_consensus.cc:385] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.169739 23977 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.169699 24286 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.169643 24285 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.170621 23977 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.170869 24285 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.172138 24286 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.172948 23977 ts_tablet_manager.cc:1434] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:51.172894 24286 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.173933 23977 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
W20260709 12:12:51.174729 23976 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.849s	user 0.016s	sys 0.040s
W20260709 12:12:51.174968 23976 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.849s	user 0.016s	sys 0.040s
I20260709 12:12:51.173476 24286 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.175379 24285 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.176461 24285 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.177482 23976 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:51.178071 24288 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.178067 23976 ts_tablet_manager.cc:1403] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 1.870s	user 0.024s	sys 0.045s
I20260709 12:12:51.177527 24285 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.178576 24288 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.180984 24286 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.183257 24285 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.183918 24285 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.183254 24286 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.184392 24285 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.186435 24285 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.186965 24285 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:12:51.191418 23826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.094s	user 0.009s	sys 0.024s
I20260709 12:12:51.187562 24285 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:51.195904 23826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.100s	user 0.011s	sys 0.026s
I20260709 12:12:51.197454 24285 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.197983 24285 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.198437 24285 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.188897 24288 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.200086 24288 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.200451 24285 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.200489 24288 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.201081 24225 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.201736 24225 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.188699 24286 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.204520 24286 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.195144 23977 tablet_bootstrap.cc:654] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.201793 24285 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.202826 24288 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.199280 23976 raft_consensus.cc:359] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.206163 23976 raft_consensus.cc:385] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.206454 23976 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.210944 23976 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.213337 24225 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.209971 24285 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.224059 23976 ts_tablet_manager.cc:1434] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.003s
I20260709 12:12:51.225044 23976 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:51.222848 24225 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.236755 24225 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.236121 24285 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.240499 24288 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.241080 24291 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.242115 24289 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.242700 24286 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.243724 24288 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.247236 23977 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:51.247821 23977 ts_tablet_manager.cc:1403] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.011s
I20260709 12:12:51.249420 24298 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.249908 24298 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.243494 24285 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.246410 24286 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.251482 24295 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.255895 24285 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.261458 23976 tablet_bootstrap.cc:654] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.244196 24291 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.267964 24225 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.248245 24289 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.313809 23977 raft_consensus.cc:359] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.314703 23977 raft_consensus.cc:385] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.314998 23977 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.313329 24225 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.349161 24298 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.349225 24225 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.257220 24295 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.349798 24289 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.366187 23977 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.366644 24289 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.367449 24289 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.367928 24295 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.368703 23977 ts_tablet_manager.cc:1434] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.121s	user 0.006s	sys 0.000s
I20260709 12:12:51.369675 23977 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:51.365914 24297 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.369789 24291 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.370409 24291 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.349815 24298 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.370289 24297 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.371995 24291 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.372069 24298 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.374488 24297 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:51.374596 23900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.869s (SENT)
I20260709 12:12:51.378603 24297 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.377276 24289 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.369823 24296 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.371184 24295 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.417505 24298 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.418169 24291 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.423352 24298 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.423300 24296 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.424083 24298 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.427042 24296 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.421902 24225 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.427558 24298 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.423357 24297 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.432838 24299 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.433468 24225 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.434134 24297 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.423728 24295 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.434679 24299 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.435901 24225 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.424880 24289 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:12:51.440667 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65251.9us
I20260709 12:12:51.442751 24295 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.443414 24295 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.443367 24297 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.440904 24289 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.447291 24225 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.448869 24289 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.450845 24300 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.444550 24295 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.444540 24297 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:51.452203 23975 raft_consensus.cc:359] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.453183 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.453658 23975 raft_consensus.cc:385] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.454005 23975 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.450414 24225 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.454764 24297 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:51.456487 23975 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.457562 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.458719 23975 ts_tablet_manager.cc:1434] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 2.125s	user 0.017s	sys 0.038s
W20260709 12:12:51.459036 23975 ts_tablet_manager.cc:1467] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Tablet startup took 2124ms
I20260709 12:12:51.460568 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 75) took 2386 ms. Trace:
I20260709 12:12:51.460704 24295 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.451687 24289 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.460994 23681 rpcz_store.cc:276] 0709 12:12:49.074111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.078465 (+  4354us) service_pool.cc:224] Handling call
0709 12:12:51.460478 (+2382013us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:12:51.464298 24299 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.461587 24289 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.469403 23977 tablet_bootstrap.cc:654] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.471509 24288 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.474159 24289 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:51.459280 23975 ts_tablet_manager.cc:1469] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Trace:
0709 12:12:43.844821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.844957 (+   136us) service_pool.cc:224] Handling call
0709 12:12:43.872991 (+ 28034us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:43.873084 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:43.934836 (+ 61752us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:43.988631 (+ 53795us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:43.989026 (+   395us) inbound_call.cc:177] Queueing success response
0709 12:12:49.298337 (+5309311us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:12:49.334161 (+ 35824us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:12:49.334658 (+   497us) tablet_replica.cc:261] Starting instrumentation
0709 12:12:51.446608 (+2111950us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":2427,"raft.queue_time_us":1500,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"tablet-open.queue_time_us":5309368}
I20260709 12:12:51.475428 23975 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:51.477694 24294 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.477950 23976 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:51.478386 23976 ts_tablet_manager.cc:1403] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.254s	user 0.013s	sys 0.012s
I20260709 12:12:51.451318 24300 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.454977 24225 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.493067 24093 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.493713 24093 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.494416 23976 raft_consensus.cc:359] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.495141 23976 raft_consensus.cc:385] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.478238 24294 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.495424 23976 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.498410 23976 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.500766 24093 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.513752 23976 ts_tablet_manager.cc:1434] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 12:12:51.514899 23976 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
W20260709 12:12:51.514921 22634 debug-util.cc:398] Leaking SignalData structure 0x7b080000f9a0 after lost signal to thread 23824
W20260709 12:12:51.515326 22634 kernel_stack_watchdog.cc:198] Thread 23824 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 418ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:12:51.510783 24093 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.520104 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.425s	user 0.031s	sys 0.069s
W20260709 12:12:51.521342 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.426s	user 0.032s	sys 0.070s
I20260709 12:12:51.526178 24300 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.535559 24307 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.536329 24307 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.538748 24307 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.520998 24093 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.541251 24288 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.548861 24309 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.549500 23977 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:51.550050 23977 ts_tablet_manager.cc:1403] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.181s	user 0.034s	sys 0.007s
I20260709 12:12:51.543730 24196 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.549124 24225 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.549464 24309 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.557965 24292 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.550287 24019 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.551486 24297 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.567334 24304 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.551347 24288 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.569583 24294 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.558938 24137 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.559813 24225 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.554337 24196 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.573050 24137 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.573850 24295 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.574002 24224 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.564297 24019 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.570533 24286 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.581139 24295 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.566823 24297 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.558555 24292 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.639566 24310 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.587299 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.640606 23860 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:51.640324 24310 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.567936 24304 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.581401 24299 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.581380 24224 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.642488 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.645273 23860 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:51.592249 23977 raft_consensus.cc:359] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.646263 23977 raft_consensus.cc:385] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.647006 23977 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
W20260709 12:12:51.597596 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800166260 after lost signal to thread 23717
I20260709 12:12:51.597589 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.651324 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.596871 24196 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.594496 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.595170 24093 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.646896 24299 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.655156 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.650795 24199 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.590197 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.595649 24309 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.594419 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.656239 24298 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.713395 23977 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.576566 24225 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.655303 24306 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.662878 24019 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.717324 24098 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.718451 24224 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.715415 24298 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.722450 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 85) took 2658 ms. Trace:
I20260709 12:12:51.733670 23683 rpcz_store.cc:276] 0709 12:12:49.063996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.064174 (+   178us) service_pool.cc:224] Handling call
0709 12:12:51.722395 (+2658221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":451527,"reported_tablets":1}
I20260709 12:12:51.698386 24137 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.666320 24199 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.724754 24201 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.727883 23856 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:51.743448 24019 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.743278 24313 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.756637 23977 ts_tablet_manager.cc:1434] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.206s	user 0.009s	sys 0.005s
I20260709 12:12:51.733917 24098 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.758461 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.758632 24286 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.759994 23975 tablet_bootstrap.cc:654] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.750967 24196 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.748983 24137 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.755154 24291 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.755699 24288 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.754344 24309 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.768276 23852 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:51.768090 24309 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.769227 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.770057 24309 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.770296 24304 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.770419 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.770833 24304 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.764667 23824 leader_election.cc:343] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.748777 24201 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.757730 24093 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.744640 24306 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.759418 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.760883 23934 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.759196 24310 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.761572 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:51.777369 23824 leader_election.cc:343] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.762992 24308 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.759366 24019 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.766288 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.771252 24304 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.771505 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.773753 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.773432 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.792026 24314 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.776803 24288 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.764845 24196 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.764207 24286 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.776134 24309 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:51.780042 24295 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.765097 24137 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.758374 24313 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.792788 24314 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.799050 24311 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:12:51.799535 24201 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.792687 24199 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.786437 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.778810 24093 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.774996 24291 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.803968 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.805490 24196 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.812213 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:51.815514 23785 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:51.816749 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:51.804033 24298 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.804055 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:51.806375 24098 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.806906 23853 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:51.817570 24299 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.807024 23930 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.801069 24288 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.856712 23975 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:51.857287 23975 ts_tablet_manager.cc:1403] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.382s	user 0.025s	sys 0.019s
I20260709 12:12:51.803231 24309 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.817374 23929 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.814438 23933 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:51.853609 23925 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:51.876026 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.877308 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.879096 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.880430 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.881505 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.882570 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.833750 23851 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:51.863476 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.866322 24313 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.858666 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.910657 23977 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:51.868786 24306 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.818156 24225 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.796139 24308 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:51.899870 23781 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:51.900666 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.906193 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.867027 24286 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.910019 24288 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.912871 23933 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:51.916760 24308 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.877916 23925 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:51.925439 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.936108 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.936823 23928 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:51.931941 24292 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.954241 24304 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.944264 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.944696 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:51.946031 24093 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:51.947108 24291 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.933521 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:51.949532 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.054286 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.055507 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.055853 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.056167 23931 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.057107 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:51.957048 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 396850us
I20260709 12:12:51.957583 23976 tablet_bootstrap.cc:654] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.954613 23681 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.959574 24285 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.960774 24297 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:51.966423 23929 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:51.968216 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:51.982321 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:51.870872 24312 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:51.994549 24092 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.995982 24310 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:51.996591 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.075265 24092 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.075309 24310 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.077548 24310 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.077593 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.078397 23860 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:51.917006 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.073410 23929 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.074508 23857 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:52.079973 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.075620 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.045885 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.081825 23928 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.081967 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.082672 23859 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:52.088591 23933 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.089562 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.090433 24309 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.084970 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.059685 24316 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.117491 24316 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.118685 23932 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.121075 24312 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:52.131728 24312 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.136005 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.136680 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.137375 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.138046 23856 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:52.138115 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.139541 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.133435 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.140285 23931 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.070696 23927 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.144560 24314 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.167632 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.168394 23854 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:52.135970 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.144626 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.152102 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.188720 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.191675 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.192492 24312 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.191713 24285 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.195888 24285 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.195686 23975 raft_consensus.cc:359] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.196286 24285 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.196429 23975 raft_consensus.cc:385] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.196754 23975 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.197402 23975 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.176357 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.199879 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.206012 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:52.207383 23824 leader_election.cc:343] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.208081 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.210952 24292 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.211319 24292 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.211457 24295 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.211784 24292 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.211871 24295 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.212761 23975 ts_tablet_manager.cc:1434] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.355s	user 0.040s	sys 0.078s
I20260709 12:12:52.213632 23751 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.214133 23975 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:52.212333 24295 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.200690 24297 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.201063 24137 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:52.203300 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800299be0 after lost signal to thread 23809
I20260709 12:12:52.199178 24312 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:52.217381 23751 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.203630 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.218120 24308 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.223274 23858 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:52.223083 24308 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.223583 24316 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.224135 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:52.224689 23784 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.224992 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.222405 23900 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.226372 24316 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.226729 24316 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.226826 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.227109 24316 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.211241 24286 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.227820 24286 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.228209 24286 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.225123 24308 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.224689 23785 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:52.229784 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.216279 24019 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.232227 23900 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.221437 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.219096 24317 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:12:52.230782 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800299de0 after lost signal to thread 23810
I20260709 12:12:52.229708 24092 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:52.235337 23824 leader_election.cc:343] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.236258 24019 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.229519 24312 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.239327 23824 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.242128 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.246682 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.239742 24019 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.247700 24019 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.248765 24296 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.248963 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.249194 24296 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.234822 24317 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:52.233904 24314 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.235580 24137 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.229405 24297 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.239297 24308 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.241093 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.272146 24105 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.272640 24296 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.235080 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:52.275065 24317 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.275578 24137 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.273216 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.277341 24308 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.274827 24314 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.282877 24105 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.291440 24105 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.291904 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:52.285660 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 50943.5us
I20260709 12:12:52.301173 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.303359 24311 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.290514 24291 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.290954 24137 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.296955 24308 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.296608 24314 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.309108 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.297482 24320 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.338428 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.339937 23824 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:52.338910 23751 leader_election.cc:343] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.312212 24291 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.303807 24311 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:52.311965 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800299680 after lost signal to thread 23884
I20260709 12:12:52.314036 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.348558 23976 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:52.349215 23976 ts_tablet_manager.cc:1403] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.834s	user 0.017s	sys 0.024s
I20260709 12:12:52.336726 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.352335 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.342571 24291 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.343899 24225 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.344293 24311 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.347232 24300 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.346822 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.345851 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.310922 24098 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260709 12:12:52.350634 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.351949 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.350991 23782 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.369393 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.370049 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.358547 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.372203 23777 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.343537 24320 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.360162 24300 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.361268 24225 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:52.381325 23977 tablet_bootstrap.cc:654] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.371402 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.398672 23784 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.403733 24137 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:52.374583 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08002043a0 after lost signal to thread 23958
I20260709 12:12:52.363066 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.416561 24332 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.425611 24314 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:52.383188 23930 consensus_meta.cc:287] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.003s
I20260709 12:12:52.375952 24227 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.426934 24308 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.397290 24019 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.433898 24332 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.390235 24300 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.453227 24332 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.453480 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.429281 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.434037 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.434896 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:52.441817 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080029c360 after lost signal to thread 24154
I20260709 12:12:52.433795 24297 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:52.459233 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 101515us
I20260709 12:12:52.438290 24227 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:52.465699 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 118937us
I20260709 12:12:52.467309 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.470090 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.451534 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.475317 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:52.477504 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49517.9us
I20260709 12:12:52.479487 24335 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.463518 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.482921 24328 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.471503 24227 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.471451 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.491622 24328 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.494542 23977 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
W20260709 12:12:52.482164 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080029c6a0 after lost signal to thread 24257
I20260709 12:12:52.453147 24225 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.493553 24331 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:12:52.471721 23781 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.530359 23975 tablet_bootstrap.cc:654] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.471158 23932 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.538275 23977 ts_tablet_manager.cc:1403] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.628s	user 0.049s	sys 0.073s
W20260709 12:12:52.541992 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080029c6c0 after lost signal to thread 24258
I20260709 12:12:52.555145 23976 raft_consensus.cc:359] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.572687 23976 raft_consensus.cc:385] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.480593 24335 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.531471 24314 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:52.575136 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 96953.2us
I20260709 12:12:52.527735 24019 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:52.552364 24325 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:12:52.576184 23976 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.533152 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.579939 23976 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.538837 24328 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.545766 24297 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 12:12:52.577450 24335 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:52.596328 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 116970us
I20260709 12:12:52.587554 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.588932 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.603257 23976 ts_tablet_manager.cc:1434] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.253s	user 0.010s	sys 0.001s
I20260709 12:12:52.579090 24333 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:12:52.591480 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.581981 24295 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:52.608881 24324 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:12:52.609449 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.610836 23931 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:52.612790 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.605448 24320 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.596387 24285 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.622206 23825 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:52.617777 24326 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:12:52.619465 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:52.627945 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.607653 23934 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:52.618498 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.631825 23977 raft_consensus.cc:359] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.643314 23977 raft_consensus.cc:385] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.643704 23977 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.645991 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.624776 24331 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.685344 24331 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:52.685747 24331 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.691582 23933 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:52.693150 24105 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.623675 24337 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.695235 24105 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.697136 24295 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.668474 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.712929 23925 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:52.689385 24336 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.644083 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.849151 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.809796 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.852367 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.818064 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.855224 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:12:52.826646 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 198587us
I20260709 12:12:52.857832 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.858124 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 127) took 1089 ms. Trace:
I20260709 12:12:52.828568 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.860280 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.860594 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.621887 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.856058 23782 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:52.861779 23931 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:52.862334 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.862582 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.860600 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.864589 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.858743 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.866679 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.867858 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:52.715410 24292 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.503s	user 0.016s	sys 0.005s
I20260709 12:12:52.865315 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.865263 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.853421 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.858640 23930 rpcz_store.cc:276] 0709 12:12:51.768741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.773948 (+  5207us) service_pool.cc:224] Handling call
0709 12:12:51.790358 (+ 16410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.469045 (+678687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.857453 (+388408us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54000f82d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:12:52.857857 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28206336}
I20260709 12:12:52.873672 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.876852 24098 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.879801 24227 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.726202 24285 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:52.734859 24286 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.004s
I20260709 12:12:52.904558 24286 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:52.881886 24291 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 12:12:52.727684 24296 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.906502 23933 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:52.768296 24300 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.685012 24297 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.909634 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.773352 24311 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.910122 24297 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:52.910305 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.910665 24297 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.868739 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:52.759179 24316 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.004s
I20260709 12:12:52.914482 24316 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.915012 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.688392 24317 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.776031 24328 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.778883 24331 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.915030 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.780618 24335 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.858723 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.917824 24317 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.898245 24098 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.902135 24292 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.904968 23975 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
W20260709 12:12:52.920318 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61900.2us
I20260709 12:12:52.904888 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.884927 23927 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:52.925940 23825 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.909958 23783 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:52.927764 24106 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:52.757305 24312 consensus_meta.cc:287] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 12:12:52.928568 24286 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.911520 23854 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:52.928776 23825 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.903117 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:52.929102 24312 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.930104 24106 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.930922 23776 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:52.912947 24291 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:52.913374 24311 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.931973 24312 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.915334 24137 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.918661 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.923277 24098 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.932628 23778 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:52.917913 24316 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.934798 24296 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.934708 24292 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.937220 23975 ts_tablet_manager.cc:1403] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.723s	user 0.013s	sys 0.014s
I20260709 12:12:52.933099 24311 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:12:52.934046 23785 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:52.933499 24291 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.933174 24344 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.944778 24344 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:52.945266 24344 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.934906 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:52.938369 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:52.952832 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:52.957116 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48737.9us
I20260709 12:12:52.963656 24308 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:52.969713 24227 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.970068 23778 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.982892 24331 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.940377 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.984478 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:52.985518 23858 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:52.985850 23860 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.986471 24328 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:52.986513 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:52.987278 23858 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.988204 23855 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.989418 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:52.991456 23855 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:52.993191 23976 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:53.001938 23857 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:53.002518 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.003340 24300 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.006063 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.007896 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.008724 23777 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:52.992892 23977 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.011147 23977 ts_tablet_manager.cc:1434] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.468s	user 0.012s	sys 0.045s
I20260709 12:12:53.012553 23977 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:52.989369 24335 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.021018 24308 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.022004 24308 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.032615 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.033317 24311 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.034648 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.039707 23932 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:53.042034 24311 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:53.042692 24311 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.016057 23858 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:53.047533 24297 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.048913 23860 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.050019 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.053206 23856 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.052842 23975 raft_consensus.cc:359] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.056627 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.057418 23859 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.058310 23860 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.064541 23857 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.068274 23931 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.070823 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.072901 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.073793 23860 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:53.073882 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.074743 23851 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.072562 24308 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:53.072072 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:12:53.079998 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145274us
I20260709 12:12:53.055661 23975 raft_consensus.cc:385] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.079519 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.083771 23852 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:53.073011 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50262us
I20260709 12:12:53.088280 24344 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.090732 24344 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.092944 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.093822 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.080809 23975 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.095688 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.096431 23932 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:53.097909 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.098371 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.099149 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.099771 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.100144 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.100737 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.101969 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.102114 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.095279 23975 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.106011 24308 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.108916 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.112375 23825 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.112188 24335 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.115983 23825 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.117214 24331 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.117785 24331 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.118151 24331 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.099200 24311 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:53.126929 23975 ts_tablet_manager.cc:1434] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.185s	user 0.008s	sys 0.001s
I20260709 12:12:53.135277 24335 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.138190 23976 tablet_bootstrap.cc:654] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.139919 23852 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:53.140417 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 61025.4us
I20260709 12:12:53.145321 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.146303 23977 tablet_bootstrap.cc:654] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.147922 23825 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.148707 23975 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:53.147940 24308 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.150614 24335 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.152942 24297 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.154261 23751 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.153425 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.160688 24288 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.161114 24288 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:53.161491 24288 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.163950 24227 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.165047 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 83) took 1656 ms. Trace:
I20260709 12:12:53.171866 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.182117 24331 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.184166 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.184437 24311 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:53.185220 24350 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.185820 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.177693 23932 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:53.191159 24308 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.198841 23977 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:53.199146 23975 tablet_bootstrap.cc:654] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.199407 23977 ts_tablet_manager.cc:1403] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.187s	user 0.011s	sys 0.005s
W20260709 12:12:53.198688 23825 leader_election.cc:343] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.201241 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.201555 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.202117 23778 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.202868 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.203068 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.205651 24311 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.207362 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.165468 23681 rpcz_store.cc:276] 0709 12:12:51.488139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.488322 (+   183us) service_pool.cc:224] Handling call
0709 12:12:53.143698 (+1655376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:52.270743 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:52.292526 (+ 21783us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:52.292561 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:12:52.292578 (+    17us) tablet.cc:662] Decoding operations
0709 12:12:52.292939 (+   361us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:52.293013 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:52.293044 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:52.293227 (+   183us) tablet.cc:701] Row locks acquired
0709 12:12:52.293249 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:12:52.293412 (+   163us) write_op.cc:270] Start()
0709 12:12:52.293523 (+   111us) write_op.cc:276] Timestamp: P: 1783599172293363 usec, L: 0
0709 12:12:52.293543 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:12:52.294252 (+   709us) log.cc:844] Serialized 537 byte log entry
0709 12:12:52.955803 (+661551us) op_driver.cc:464] REPLICATION: finished
0709 12:12:53.106200 (+150397us) write_op.cc:301] APPLY: starting
0709 12:12:53.106318 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:53.106448 (+   130us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:53.107933 (+  1485us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:53.108339 (+   406us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:53.108606 (+   267us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:12:53.108678 (+    72us) write_op.cc:312] APPLY: finished
0709 12:12:53.109083 (+   405us) log.cc:844] Serialized 22 byte log entry
0709 12:12:53.109348 (+   265us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.109473 (+   125us) write_op.cc:454] Released schema lock
0709 12:12:53.109525 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:53.109976 (+   451us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.110009 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":149624,"apply.run_cpu_time_us":2592,"apply.run_wall_time_us":3744,"num_ops":1,"prepare.queue_time_us":160435,"prepare.run_cpu_time_us":33941,"prepare.run_wall_time_us":126274,"raft.queue_time_us":347553,"raft.run_cpu_time_us":110487,"raft.run_wall_time_us":128024,"replication_time_us":661546,"thread_start_us":258071,"threads_started":4,"wal-append.queue_time_us":136751}]]}
I20260709 12:12:53.211733 23977 raft_consensus.cc:359] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.212076 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.213167 23932 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:53.213805 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.214609 24344 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.215781 24311 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.220057 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.221964 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.223745 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.224172 23825 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.212558 23977 raft_consensus.cc:385] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.227921 23977 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.205695 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.231556 23778 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:53.232395 24311 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.232792 24311 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.233093 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.221951 24344 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.233116 24311 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.237988 24300 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.238487 24300 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.253508 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.255065 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.261981 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.254386 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.252764 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.263844 23852 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.251636 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.254493 24291 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.264982 24291 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.247017 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.248806 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.269352 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.246636 24300 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.229856 24288 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.263856 23927 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.279047 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.280119 23926 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:53.254772 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.281509 23751 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.282456 24327 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.282825 24327 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.283150 24327 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.267418 24291 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.255440 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.269245 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.284878 24312 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:12:53.290782 24137 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:53.291568 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.300081 23932 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:53.300026 23855 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.292842 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.301329 23925 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:53.301667 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.301824 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.302177 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.302780 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.245102 24331 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.295908 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.304752 24291 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.305174 24137 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.305565 24137 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.283751 24098 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:12:53.306607 24098 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.306545 24137 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.307739 24098 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.294970 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:53.303416 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.305166 24291 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.312572 24291 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.313671 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.315697 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.310143 24098 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.319072 24098 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:53.319695 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 79765.2us
I20260709 12:12:53.309998 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.313899 24291 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.320894 23977 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.321462 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.322170 24137 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:53.299358 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:53.309172 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.328347 24300 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.330209 24288 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.330283 24300 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:53.330749 24300 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.308868 24311 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.329001 23751 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.334606 24311 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.339787 23927 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:53.340296 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:53.340745 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:53.341387 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.342171 24317 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.342185 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:53.345026 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.327559 24350 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.345973 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.345923 24288 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:53.346693 23777 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:53.344959 23779 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:53.347545 24317 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.348006 24311 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:12:53.348768 23749 leader_election.cc:343] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.348968 24317 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.345638 23926 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.342504 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:53.351112 23784 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.351315 24288 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.351682 24288 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.352507 23975 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:53.355684 23975 ts_tablet_manager.cc:1403] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.207s	user 0.023s	sys 0.005s
W20260709 12:12:53.356830 23824 leader_election.cc:343] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.357249 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.358686 23932 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.362165 23977 ts_tablet_manager.cc:1434] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.162s	user 0.011s	sys 0.001s
I20260709 12:12:53.366204 23975 raft_consensus.cc:359] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.366914 23975 raft_consensus.cc:385] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.367137 23975 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.356164 24288 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.369488 23975 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.373198 23975 ts_tablet_manager.cc:1434] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:12:53.373696 23977 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:53.374323 23975 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:53.376529 23976 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:53.376930 23976 ts_tablet_manager.cc:1403] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.384s	user 0.009s	sys 0.028s
I20260709 12:12:53.378149 24331 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.361824 24311 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.388597 23751 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.394510 24329 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.394961 24329 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.395280 24329 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.377758 24291 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:12:53.382056 24300 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.390455 24350 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:12:53.401697 24285 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:12:53.381033 24312 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:12:53.386132 24286 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:12:53.401695 24098 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:12:53.402032 24288 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.406550 23855 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.408154 24320 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:12:53.412369 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.412143 24316 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:53.412596 24320 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:53.413266 24316 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.413645 24316 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:53.413569 24320 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.414633 24316 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.417466 24316 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.426689 23855 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.428989 24308 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:53.430066 24308 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.430430 24308 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:53.431099 23680 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.429291 24329 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.435662 24300 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.446441 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:53.451479 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.450115 24312 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.475412 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.466747 24317 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.485134 24317 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.466830 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.464165 24098 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.462493 24312 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.466336 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.486188 24317 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.506110 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.486927 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.486250 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.485458 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.486315 24098 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.486128 24329 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.485702 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.457842 24308 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:53.494741 23927 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:53.518776 23751 leader_election.cc:343] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.515781 24285 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:53.519804 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 142908us
W20260709 12:12:53.526911 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 122964us
I20260709 12:12:53.541659 23932 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:53.542546 23977 tablet_bootstrap.cc:654] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.544490 24363 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.544963 24363 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.545960 24363 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.547263 24363 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.547770 23855 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.552839 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:53.564599 23824 leader_election.cc:343] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.565346 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:53.566699 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.569360 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.551081 23852 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.581172 23926 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:53.585367 24333 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:53.548080 24308 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.596385 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.599853 24285 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.602442 23975 tablet_bootstrap.cc:654] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.602908 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:12:53.604208 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.551287 24098 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.611387 23976 raft_consensus.cc:359] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.614106 23976 raft_consensus.cc:385] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.614442 23976 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.615377 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.572600 24316 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:12:53.616711 24367 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.618669 24137 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.601173 24333 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:53.621147 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.622013 24288 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.622641 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:53.623026 24362 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.624094 24320 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.625027 24098 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.625844 23749 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:53.626286 23824 leader_election.cc:343] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.627112 24333 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:53.631979 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72520.6us
I20260709 12:12:53.567889 24312 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.606907 24366 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:53.619064 24137 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.637490 24137 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.634357 24288 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.638742 24137 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.632989 23976 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.634227 24327 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.640637 24312 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.657485 23852 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:53.659461 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.651227 24327 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.653280 24316 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.666249 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.667907 23976 ts_tablet_manager.cc:1434] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.291s	user 0.006s	sys 0.002s
I20260709 12:12:53.670877 24286 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.671307 24286 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.658587 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.655896 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.673991 24366 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.659904 23927 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:53.666827 24316 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:53.692539 24316 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.693877 24316 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.667622 24363 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:12:53.685688 23976 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:53.695911 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.697099 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.698202 23749 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.689358 23855 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:53.690219 24333 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.700049 23899 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.691280 23931 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:53.701639 24333 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.704152 24367 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.715517 24098 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.717243 24098 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:53.717005 24367 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.717754 24098 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.715446 24327 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.717587 24373 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.719249 24367 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:53.705179 24366 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.720844 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.722004 24320 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.722299 23855 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:53.723107 24286 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.724009 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:53.724328 24286 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.724798 24333 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.720139 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.725517 24366 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.725862 24366 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.726193 24366 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.726331 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.727182 24327 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:53.729524 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55908.8us
I20260709 12:12:53.730361 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.730885 23824 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:53.735564 23824 leader_election.cc:343] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.735994 23977 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:53.736488 23977 ts_tablet_manager.cc:1403] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.367s	user 0.019s	sys 0.012s
I20260709 12:12:53.737957 24345 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.736459 24333 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.740054 24333 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:53.737098 24320 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.740873 23977 raft_consensus.cc:359] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.741688 23977 raft_consensus.cc:385] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.740868 24333 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.741973 23977 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.742632 23977 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.745335 24366 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.748167 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.748769 23977 ts_tablet_manager.cc:1434] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 12:12:53.748867 23824 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:53.749970 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.750233 23976 tablet_bootstrap.cc:654] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.749959 23977 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:53.750952 24345 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:53.753911 24345 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:53.754837 24345 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:53.751538 23899 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.760375 24373 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:53.760874 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.761715 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.765009 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.764019 24373 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.770694 23975 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:53.771240 23975 ts_tablet_manager.cc:1403] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.397s	user 0.013s	sys 0.005s
I20260709 12:12:53.771908 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:12:53.773588 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.775880 24137 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.776273 24137 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.774295 24373 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.777673 24137 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.783190 23899 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.784065 24373 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.785194 23899 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.785933 24373 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.787470 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.788211 24373 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.787918 24137 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.791234 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.792954 23975 raft_consensus.cc:359] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.793655 23975 raft_consensus.cc:385] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.793910 23975 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.794552 23975 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.778208 24366 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.800621 24098 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.805673 23977 tablet_bootstrap.cc:654] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.806850 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.801920 24137 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.819769 23975 ts_tablet_manager.cc:1434] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:12:53.821053 23975 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:53.823751 24286 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.827050 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.828243 24327 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.828851 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.829353 24366 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.838871 24098 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.834515 24286 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.833683 24366 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.840965 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.839516 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.855530 23976 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:53.856073 23976 ts_tablet_manager.cc:1403] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.174s	user 0.013s	sys 0.014s
I20260709 12:12:53.856361 24366 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.857729 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:53.859742 23751 leader_election.cc:343] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.848476 23977 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:53.852385 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.861078 23977 ts_tablet_manager.cc:1403] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.111s	user 0.008s	sys 0.004s
I20260709 12:12:53.861248 23975 tablet_bootstrap.cc:654] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.862054 24098 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.862423 24098 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.862334 23976 raft_consensus.cc:359] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.862849 24098 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.863078 23976 raft_consensus.cc:385] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.863418 23976 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.864362 23976 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.867553 23976 ts_tablet_manager.cc:1434] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:12:53.876403 23976 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:53.880635 24366 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.877063 23977 raft_consensus.cc:359] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.884310 23977 raft_consensus.cc:385] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.884872 23977 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.885591 23977 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.875010 24098 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.887240 23683 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.888727 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.890172 23977 ts_tablet_manager.cc:1434] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.004s
I20260709 12:12:53.892203 24098 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:53.891242 23977 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:53.862936 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.899627 24098 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.900141 23749 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.900907 24098 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.901106 23975 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:53.901764 23975 ts_tablet_manager.cc:1403] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.081s	user 0.012s	sys 0.001s
I20260709 12:12:53.901912 23977 tablet_bootstrap.cc:654] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.904505 23975 raft_consensus.cc:359] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.905223 23975 raft_consensus.cc:385] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.905488 23975 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.906181 23975 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.892716 23976 tablet_bootstrap.cc:654] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:53.907702 23749 leader_election.cc:343] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.908113 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.909224 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.909956 24366 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.910246 23749 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.910581 24366 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.912458 23975 ts_tablet_manager.cc:1434] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:12:53.913502 23975 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:53.914021 24316 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.914476 24316 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.916538 24316 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:53.918922 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:53.919631 23852 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:53.919982 23977 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:53.920437 23977 ts_tablet_manager.cc:1403] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.006s
I20260709 12:12:53.921198 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:53.921960 24316 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.922339 24316 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.922663 24316 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.923118 23977 raft_consensus.cc:359] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.923807 23977 raft_consensus.cc:385] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.924072 23977 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.925508 24286 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.927944 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.928735 24286 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.930097 23749 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.935650 24098 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:12:53.936882 23749 leader_election.cc:343] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.937829 23749 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.924911 23977 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.934211 23681 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.940119 23977 ts_tablet_manager.cc:1434] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:12:53.940543 24098 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.941282 23977 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:53.941895 23749 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.944003 24098 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.945385 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.942766 23681 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.946970 23681 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.948086 23977 tablet_bootstrap.cc:654] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.948153 24098 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.949249 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.950390 24316 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.951994 24098 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:12:53.955360 23749 leader_election.cc:343] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.956418 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.957696 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:53.958283 24366 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.962921 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:53.963351 24098 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:53.963542 23855 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.963903 24098 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:53.964795 24098 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.971400 23855 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:53.973053 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:53.973408 24316 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:53.974140 24316 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.978164 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:53.978951 24098 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:12:53.980135 23749 leader_election.cc:343] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.980611 23976 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:53.981153 23976 ts_tablet_manager.cc:1403] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.105s	user 0.002s	sys 0.012s
I20260709 12:12:53.984484 24316 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:53.992225 24316 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:53.980552 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.000322 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.001425 24286 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.999181 23682 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 2, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.006933 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.007773 23777 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:54.008316 23784 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:53.994274 23976 raft_consensus.cc:359] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.010051 23976 raft_consensus.cc:385] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.012240 23976 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.013684 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.014863 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.021950 23784 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.022042 23777 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.023347 23977 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:53.997011 23975 tablet_bootstrap.cc:654] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.023312 23976 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.023948 23977 ts_tablet_manager.cc:1403] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.083s	user 0.032s	sys 0.007s
I20260709 12:12:54.025553 23976 ts_tablet_manager.cc:1434] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.001s
I20260709 12:12:54.026360 23976 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:54.026799 23977 raft_consensus.cc:359] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.027060 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.027637 23977 raft_consensus.cc:385] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.027985 23977 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.028705 23977 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.029261 23784 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:54.028070 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.030964 23977 ts_tablet_manager.cc:1434] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:12:54.032425 23977 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.033974 23976 tablet_bootstrap.cc:654] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.037436 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.038317 23785 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:54.038421 23977 tablet_bootstrap.cc:654] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.043571 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.044849 23976 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:54.045342 23976 ts_tablet_manager.cc:1403] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.005s
I20260709 12:12:54.047945 24098 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.048231 23976 raft_consensus.cc:359] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.048910 23976 raft_consensus.cc:385] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.049178 23976 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.050483 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.050449 23976 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.053381 23976 ts_tablet_manager.cc:1434] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:12:54.053812 24098 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.054497 23976 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:54.055949 24098 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.057651 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.058125 24344 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.058387 23855 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:54.058586 24344 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.059742 23751 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:54.052680 23680 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.060503 24098 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.060726 24344 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.062696 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.063342 23928 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:54.064776 23824 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.065569 24344 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.065924 24344 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.060505 23975 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:54.066278 23976 tablet_bootstrap.cc:654] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.061138 24098 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.066684 24098 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.069893 23977 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.070504 23977 ts_tablet_manager.cc:1403] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.005s
I20260709 12:12:54.073185 23977 raft_consensus.cc:359] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.073843 23977 raft_consensus.cc:385] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.074110 23977 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.074142 24098 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.075037 23977 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.075846 23975 ts_tablet_manager.cc:1403] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.000s
I20260709 12:12:54.066246 24344 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.083281 23977 ts_tablet_manager.cc:1434] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:12:54.084751 23977 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.084442 23975 raft_consensus.cc:359] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.085331 23975 raft_consensus.cc:385] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.085665 23975 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.091641 23977 tablet_bootstrap.cc:654] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.095907 23975 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.057847 23785 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.098663 24098 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.100229 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.103621 23976 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:54.103660 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:54.104207 23976 ts_tablet_manager.cc:1403] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.050s	user 0.025s	sys 0.018s
I20260709 12:12:54.104408 23855 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.105950 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.107828 23778 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:54.108078 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.110164 24225 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.110661 24225 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.112826 24225 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.113300 23855 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:54.113969 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.114303 23781 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.114675 23751 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:12:54.115542 23824 leader_election.cc:343] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.116044 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.115352 23976 raft_consensus.cc:359] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.116698 23976 raft_consensus.cc:385] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.116978 23976 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.117691 23976 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.119908 23976 ts_tablet_manager.cc:1434] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:12:54.122836 23977 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.122469 24344 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.123303 23977 ts_tablet_manager.cc:1403] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.000s
I20260709 12:12:54.123767 24098 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.125473 24098 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:54.126259 24098 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.126457 23977 raft_consensus.cc:359] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.127111 23977 raft_consensus.cc:385] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.127375 23977 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.127341 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.128019 23928 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.128180 23977 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.129737 23977 ts_tablet_manager.cc:1434] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:54.130654 23977 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.136375 23977 tablet_bootstrap.cc:654] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.148303 23927 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:54.151193 23928 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:54.125422 23976 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:54.153752 24345 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:54.152006 24344 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.158943 23976 tablet_bootstrap.cc:654] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.160094 23977 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.160578 23977 ts_tablet_manager.cc:1403] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 12:12:54.160899 23824 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.162025 24344 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.162510 24344 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.163359 24344 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.163714 23977 raft_consensus.cc:359] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.164343 23977 raft_consensus.cc:385] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.164618 23977 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.165257 23977 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.167582 23977 ts_tablet_manager.cc:1434] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:12:54.168663 23977 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.185132 23777 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.187368 23776 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:54.190761 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.192649 23776 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:54.194674 23680 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.195693 23785 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.197075 23975 ts_tablet_manager.cc:1434] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.120s	user 0.005s	sys 0.002s
I20260709 12:12:54.200502 23975 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:54.202909 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.203895 23785 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:54.204675 23779 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:54.207402 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.213174 23681 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.215659 23977 tablet_bootstrap.cc:654] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.212672 23776 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:54.218753 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.219560 23781 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:54.222956 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.230086 23977 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.230753 23977 ts_tablet_manager.cc:1403] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.062s	user 0.010s	sys 0.010s
I20260709 12:12:54.231446 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.235953 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.236851 24344 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.237375 24344 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.237816 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.239483 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.239704 24316 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.244418 23785 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.236867 24373 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.248857 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.249931 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.250526 23927 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.251446 23933 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.251746 23899 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:12:54.252422 23824 leader_election.cc:343] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.252910 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.253731 23855 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:54.253925 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.251361 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.255141 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.255234 24316 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:54.255950 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.260015 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.261134 24316 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:54.261590 24316 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:54.264496 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.264750 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.265818 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.266269 23927 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:54.267496 24098 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:54.268258 23931 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:54.265854 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.269299 23785 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:54.268658 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.268719 24367 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.268857 24373 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.266510 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.277685 23784 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:54.272303 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.272744 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.273173 23778 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:54.281682 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.282447 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.280071 24333 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.284723 23781 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:12:54.285125 23781 pending_rounds.cc:85] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 0
I20260709 12:12:54.285462 23781 pending_rounds.cc:107] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:12:54.285790 23781 raft_consensus.cc:2889] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:54.276690 24098 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.287367 24098 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:54.288122 24098 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.288502 23931 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:54.289130 24098 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.290246 24333 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.276940 23975 tablet_bootstrap.cc:654] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.273248 23782 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:54.239502 24344 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.257256 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.271474 23777 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.279479 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.256254 24308 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.280819 23783 raft_consensus.cc:2368] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2.
W20260709 12:12:54.301929 23749 leader_election.cc:343] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.281492 23932 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:54.286660 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.302453 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.305796 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.303099 24286 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.307830 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:54.305984 23899 leader_election.cc:343] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.314813 23899 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.316383 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.307047 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.311714 24345 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.318852 24345 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.319303 24373 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.320732 24373 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.320327 24345 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.312244 24364 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.321722 24364 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.323355 24316 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.323668 23977 raft_consensus.cc:359] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.324373 23977 raft_consensus.cc:385] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.325373 24316 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.324631 23977 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
W20260709 12:12:54.335956 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56519.1us
I20260709 12:12:54.337432 24098 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.338389 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.339107 24098 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.336558 24286 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.342908 24286 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.343219 23777 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:54.341058 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.344825 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.304369 24366 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.345458 23855 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:54.314152 24308 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:12:54.347054 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:54.347178 24316 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.307241 24327 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.347721 24316 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:54.348474 24366 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.349004 23976 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:54.348685 24316 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.349421 23976 ts_tablet_manager.cc:1403] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.224s	user 0.030s	sys 0.032s
I20260709 12:12:54.339584 23977 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.351975 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.352190 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.323807 24364 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.354504 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.355124 24098 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.361631 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:54.363940 23899 leader_election.cc:343] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.350164 24366 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:54.367306 23749 leader_election.cc:343] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.372124 23749 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.360646 24286 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.362162 24331 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.373449 24225 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.373613 24327 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.375996 24098 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:54.376626 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:54.371729 23899 leader_election.cc:400] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2.
I20260709 12:12:54.374699 23977 ts_tablet_manager.cc:1434] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.143s	user 0.007s	sys 0.000s
I20260709 12:12:54.368263 24316 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.375664 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.377115 23899 leader_election.cc:403] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:54.369565 24366 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.378084 23975 tablet_bootstrap.cc:492] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:54.378448 23975 ts_tablet_manager.cc:1403] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.178s	user 0.026s	sys 0.023s
I20260709 12:12:54.379211 24331 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.380160 23977 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.380637 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.381352 23855 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:54.364920 24367 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:12:54.376209 24225 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.379727 24327 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:54.383270 24411 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:54.383535 24327 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.383795 24411 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.383723 24331 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.385847 24411 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.383038 24320 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.386572 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.386919 24320 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:54.387777 24320 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.382779 24225 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.390029 23976 raft_consensus.cc:359] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.390681 23976 raft_consensus.cc:385] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:12:54.390878 23749 leader_election.cc:343] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.382681 24409 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.392508 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:54.389082 24320 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.393800 23975 raft_consensus.cc:359] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.394670 23975 raft_consensus.cc:385] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.394976 23975 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.390930 23976 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.398312 24344 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.405236 24407 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.405807 24407 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.407294 23976 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.408129 23975 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.408937 24098 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:54.408164 24407 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:54.411087 23749 leader_election.cc:343] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.412077 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.412454 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.412796 23932 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:54.413048 23780 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:54.416339 23976 ts_tablet_manager.cc:1434] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:12:54.417378 23976 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:54.419903 23824 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.421072 24344 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.421461 24344 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.421782 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.421782 24344 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.427577 24366 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.429798 24366 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.429390 23681 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.432066 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.432894 23777 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:54.433602 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.437347 23932 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:54.439672 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:54.440119 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.440317 23855 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.443434 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.448195 24286 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.448608 24286 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:54.448721 23852 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:54.449426 24286 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.450697 24286 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.442412 24411 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.452013 24411 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.452374 24411 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.454596 23680 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:54.456780 23749 leader_election.cc:343] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.455762 24098 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:12:54.458235 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82654.1us
I20260709 12:12:54.406884 24409 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2.
I20260709 12:12:54.466189 23825 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.463899 23900 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.467721 23900 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.469403 24316 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:12:54.470112 23977 tablet_bootstrap.cc:654] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.472666 24345 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.467515 24417 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:12:54.471747 23825 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:54.493500 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:54.493671 24367 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:54.502185 24320 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:54.502323 24367 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.504761 23976 tablet_bootstrap.cc:654] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.507118 23975 ts_tablet_manager.cc:1434] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.128s	user 0.021s	sys 0.011s
I20260709 12:12:54.508664 24367 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.509519 23975 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:54.509686 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.510087 24407 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.475442 24344 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.465540 24331 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:12:54.519805 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.482190 24409 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.536023 23777 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:54.561020 23899 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:54.549921 23855 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:54.563736 24316 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.551820 24422 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.564385 24422 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.565212 24422 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.566493 24422 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.574688 24344 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:54.579571 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68695.8us
I20260709 12:12:54.581458 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:54.590821 23749 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:54.592540 24419 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.593375 24419 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.593698 24419 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.594295 23975 tablet_bootstrap.cc:654] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.597211 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.595672 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.588307 24225 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.605955 24411 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.609308 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.600533 24418 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.622901 24327 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:12:54.611850 24419 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.626459 24419 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.627564 24424 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.628579 24424 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.606105 24316 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.616248 24409 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.608776 24225 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.628737 24286 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:12:54.628454 24344 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.638200 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:54.638753 24411 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.639078 23825 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.639734 24423 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.626480 23680 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.642720 23927 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.643078 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:54.643675 23857 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.643779 24423 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:54.643893 23977 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.644142 24423 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.644433 23977 ts_tablet_manager.cc:1403] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.265s	user 0.012s	sys 0.015s
I20260709 12:12:54.644973 24423 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.646235 24423 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.648459 24286 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.650653 23927 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:54.652194 23680 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.647076 23977 raft_consensus.cc:359] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.655606 23977 raft_consensus.cc:385] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.655992 23977 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.660205 23824 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.629168 24424 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.631397 24418 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:12:54.670217 23976 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:54.664628 24344 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.670476 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:54.670754 23976 ts_tablet_manager.cc:1403] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.254s	user 0.017s	sys 0.011s
I20260709 12:12:54.660310 24225 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.672181 23780 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.672360 24344 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.671242 24331 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.683804 24418 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.689261 24225 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.675518 23976 raft_consensus.cc:359] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.691067 23976 raft_consensus.cc:385] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.691394 23976 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.692093 23976 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.694324 23976 ts_tablet_manager.cc:1434] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.003s
I20260709 12:12:54.694465 24331 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.694802 24424 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.693771 23977 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.695334 23976 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:54.679306 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.695472 24333 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.711405 23977 ts_tablet_manager.cc:1434] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:12:54.714267 23825 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.710933 24424 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.723882 24422 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:54.695334 24331 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.724629 24419 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.725261 24333 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.707551 24407 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.734439 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.735880 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.726197 24333 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.738147 23977 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:54.732000 24327 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.735140 24422 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.736609 24345 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.754071 23825 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.743578 24419 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:54.757390 23931 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:12:54.740728 24327 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.770016 24345 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:54.770814 24316 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.770895 24345 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:54.771320 24345 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:54.771351 24316 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.766943 24428 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:12:54.756321 24429 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.779918 24316 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:54.756817 23857 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:54.809545 23975 tablet_bootstrap.cc:492] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:54.810176 23975 ts_tablet_manager.cc:1403] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.301s	user 0.014s	sys 0.028s
W20260709 12:12:54.811096 24433 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.104s	user 0.012s	sys 0.000s
I20260709 12:12:54.772205 24345 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.819948 23680 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.823949 24430 raft_consensus.cc:2764] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:12:54.825153 24327 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.825943 23977 tablet_bootstrap.cc:654] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.826432 23780 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:54.826769 24345 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:54.819327 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.828725 23777 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:54.813417 24418 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.813879 24316 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.782073 24407 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.835160 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.819011 24422 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.854066 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:54.837436 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.840490 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.836890 24418 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.857020 24320 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.861786 24320 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:54.862972 23976 tablet_bootstrap.cc:654] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.865043 24418 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.827504 24437 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 12:12:54.856246 24409 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:54.867033 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.868880 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.868898 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:54.872291 24286 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:54.873430 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.876425 24286 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.877169 24286 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.862767 24320 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.836910 24316 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.867125 24409 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.913292 24409 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:54.912676 23975 raft_consensus.cc:359] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.917399 23975 raft_consensus.cc:385] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.917748 23975 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.895836 24286 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:12:54.928486 23975 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:54.930281 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 173302us
W20260709 12:12:54.941272 23825 leader_election.cc:343] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.941996 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.919920 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:54.942762 24443 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.945155 23975 ts_tablet_manager.cc:1434] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.135s	user 0.004s	sys 0.004s
W20260709 12:12:54.947117 23825 leader_election.cc:343] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.948248 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:54.942562 24407 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
W20260709 12:12:54.982187 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.988242 23976 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:54.988756 23976 ts_tablet_manager.cc:1403] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.294s	user 0.009s	sys 0.017s
I20260709 12:12:54.989805 23977 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:54.990253 23977 ts_tablet_manager.cc:1403] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.257s	user 0.015s	sys 0.011s
I20260709 12:12:54.990329 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:54.991963 24440 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.992688 24440 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.994388 23777 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.994779 24440 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:54.994915 24409 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:54.996292 23780 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:54.998399 23927 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.971187 23859 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.009649 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:55.011695 23782 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.011989 23975 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:55.014518 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:55.015130 23785 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:55.016795 23824 leader_election.cc:343] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.983706 24333 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.019770 24424 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.025951 23976 raft_consensus.cc:359] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.026814 23976 raft_consensus.cc:385] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.027101 23976 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.045531 23785 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:55.051019 24345 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:55.054309 23976 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.023838 24440 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.056332 23977 raft_consensus.cc:359] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.057140 23977 raft_consensus.cc:385] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.057435 23977 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.033383 24419 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.060602 24419 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.061220 23934 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.062820 24419 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.062975 23930 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.030366 24424 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.032590 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.058133 23977 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.058683 24437 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:12:55.069655 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:55.070927 23976 ts_tablet_manager.cc:1434] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.082s	user 0.002s	sys 0.005s
I20260709 12:12:55.071492 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.072225 23785 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:55.087234 24409 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.089620 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:55.090391 23776 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:55.092579 23824 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.094991 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.095543 23977 ts_tablet_manager.cc:1434] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.104s	user 0.006s	sys 0.002s
I20260709 12:12:55.085041 24437 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.103439 23825 leader_election.cc:343] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.103910 23825 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:55.105538 23977 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Bootstrap starting.
I20260709 12:12:55.066892 24440 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.111925 24423 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.112900 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.103402 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:55.118994 23975 tablet_bootstrap.cc:654] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.094782 24331 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:12:55.097916 24225 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:12:55.119833 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.106210 24316 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:55.142150 23825 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.142627 23825 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:55.104877 24437 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.129279 23852 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:55.155305 24424 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.157226 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:55.159183 23824 leader_election.cc:343] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.153657 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:55.161520 23825 leader_election.cc:343] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.161906 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:55.163246 24225 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.165946 24424 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:55.166241 23927 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.133555 23859 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:55.147987 24411 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.148715 24423 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.180512 24411 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.208895 23976 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:55.188330 24409 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.210363 24448 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:12:55.204182 24419 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:12:55.188526 24225 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:12:55.209712 24411 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.236661 23776 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:55.237979 23925 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:12:55.238409 24455 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:12:55.221864 24409 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:55.245445 24451 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:12:55.257009 24316 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.262552 23977 tablet_bootstrap.cc:654] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:55.268155 24452 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:12:55.253798 24440 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.257588 24316 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.284960 24458 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:55.285679 24458 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.294016 24458 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.297461 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.301841 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.302527 23784 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
W20260709 12:12:55.373129 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68968.9us
I20260709 12:12:55.273793 24320 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:12:55.374099 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 67294.7us
W20260709 12:12:55.402127 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98517.6us
I20260709 12:12:55.431537 23825 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:55.318007 23780 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.437132 24430 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.437568 24430 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.437907 24430 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.291472 23852 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:55.440853 23976 tablet_bootstrap.cc:654] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.410714 24458 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.400636 24225 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:12:55.443344 23779 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.436022 24320 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.411969 24409 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.448074 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.449599 23785 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.449862 24437 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.453197 23681 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:55.476970 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:55.466279 24458 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.530679 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:55.531756 24366 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:55.532312 24366 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.534425 24366 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:55.556308 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 461347us
I20260709 12:12:55.546787 24316 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.564689 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.559005 24411 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.595952 23977 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: No bootstrap required, opened a new log
I20260709 12:12:55.597409 23977 ts_tablet_manager.cc:1403] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Time spent bootstrapping tablet: real 0.498s	user 0.029s	sys 0.042s
I20260709 12:12:55.601117 23975 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:55.607266 23975 ts_tablet_manager.cc:1403] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.600s	user 0.011s	sys 0.027s
I20260709 12:12:55.577868 24316 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:55.589888 24467 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.639299 24316 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:12:55.730419 24461 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:12:55.718940 24409 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.733031 23977 raft_consensus.cc:359] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.726938 24331 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:55.733881 23977 raft_consensus.cc:385] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.734217 23977 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.734875 23977 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.640779 24467 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.744810 24411 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.745749 24467 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.751168 24316 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.756649 24466 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:55.757232 24466 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.727382 24320 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.766089 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.767323 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.767668 23975 raft_consensus.cc:359] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.768036 23777 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:12:55.768362 23975 raft_consensus.cc:385] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.768640 23975 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.769300 23975 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.764025 24366 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.770736 24366 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.771544 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.772223 23780 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:55.772887 24366 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.763221 24286 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.773619 24286 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.775761 23784 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:55.776338 24466 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.777846 23976 tablet_bootstrap.cc:492] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:55.778345 23976 ts_tablet_manager.cc:1403] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.576s	user 0.020s	sys 0.011s
I20260709 12:12:55.759757 24409 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.780345 24409 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.782578 24430 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.784113 23976 raft_consensus.cc:359] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.786461 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:55.784776 24430 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.787111 23926 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:55.785223 23976 raft_consensus.cc:385] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.787716 23976 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.788743 24286 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.789418 24409 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:55.788897 23976 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.811643 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.812356 23785 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.812625 23976 ts_tablet_manager.cc:1434] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:12:55.813187 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.813675 23976 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:55.813846 23779 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.816653 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.818238 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.819164 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.820679 23780 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:55.789918 24409 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.818979 23975 ts_tablet_manager.cc:1434] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.207s	user 0.006s	sys 0.000s
I20260709 12:12:55.823778 23777 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:55.830394 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.831280 23859 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:55.833884 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.834100 24419 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:55.823078 24409 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:55.833894 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.834803 23857 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.840387 23855 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.839702 23976 tablet_bootstrap.cc:654] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:55.833933 23825 leader_election.cc:343] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.759374 24331 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.837826 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.850471 23975 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:12:55.846517 23780 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.851162 23854 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:55.855960 23977 ts_tablet_manager.cc:1434] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Time spent starting tablet: real 0.258s	user 0.008s	sys 0.016s
I20260709 12:12:55.851876 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.856564 24320 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:55.856935 24320 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.834801 24443 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:12:55.857261 24320 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:55.858590 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.855317 23933 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.850096 24467 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:55.836970 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:55.847827 24286 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:55.847481 24333 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:55.846575 23783 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.846923 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:55.873050 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:55.864436 24462 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:55.834667 24345 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:12:55.876902 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:55.862464 24429 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:55.877854 23785 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.873445 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.872560 24466 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.874578 23858 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.871876 24286 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:55.871517 24467 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.875375 24333 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.888729 23780 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.882477 24462 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.880206 24411 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:55.878088 24331 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:55.893567 24464 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.901546 24464 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.900347 24411 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:55.902776 24411 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:55.896389 23778 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:55.888466 24466 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:55.899233 23824 leader_election.cc:343] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.907899 23824 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:12:55.880651 23934 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:55.889704 24286 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:55.890404 24316 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:55.912362 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 167) took 1015 ms. Trace:
I20260709 12:12:55.882848 24429 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.901001 23851 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:56.314640 24417 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.925832 24333 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.478904 24484 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.478765 24417 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.479501 24484 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.480052 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:56.480842 23776 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.481050 24493 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.931856 24345 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.481571 24493 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.964445 23785 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:56.470398 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 571136us
I20260709 12:12:56.484369 24491 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.900666 24462 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.090793 24320 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.482106 24485 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.475836 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:55.952991 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.484473 24488 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.953992 24327 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:56.223196 24480 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:12:56.426640 24286 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.001s
I20260709 12:12:55.925485 24366 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.509644 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.518261 23785 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:12:56.377447 24477 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:12:56.085939 24308 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.476297 24486 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.954967 24464 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.289440 24423 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.952827 24409 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.086302 24331 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.468381 24424 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.951902 23858 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:56.468186 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 261) took 1165 ms. Trace:
I20260709 12:12:56.085928 24428 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.103283 24475 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.936434 24418 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.952035 24458 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.469714 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 170) took 1041 ms. Trace:
I20260709 12:12:56.110235 24367 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.528626 24464 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.953387 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.485303 24466 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.486917 24430 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.527005 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:56.503643 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:55.939991 24316 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.385372 24411 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.925694 24474 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.685s
W20260709 12:12:56.539300 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:56.540504 23975 tablet_bootstrap.cc:654] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.926959 24467 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.547487 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.521044 24286 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.552512 23785 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:56.516372 24327 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.522353 24366 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.526988 24308 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.559978 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.560779 23783 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:56.533907 24424 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.562978 24424 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.545974 24418 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.564416 24320 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.513350 24488 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.566613 23778 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:56.503506 24491 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.568799 24411 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.532886 24486 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.569306 23855 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:56.545991 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 262) took 1243 ms. Trace:
I20260709 12:12:56.532680 24423 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.534125 24409 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.541991 24331 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.532503 23928 rpcz_store.cc:276] 0709 12:12:55.302742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.318006 (+ 15264us) service_pool.cc:224] Handling call
0709 12:12:55.409067 (+ 91061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:55.534682 (+125615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.534715 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:55.535224 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:12:55.636652 (+101428us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:55.638115 (+  1463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.426775 (+788660us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400202bd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:12:56.426834 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:56.426888 (+    54us) raft_consensus.cc:1685] finished
0709 12:12:56.426905 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:56.468087 (+ 41182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2791,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"spinlock_wait_cycles":265542016,"thread_start_us":78373,"threads_started":1,"wal-append.queue_time_us":197114}
I20260709 12:12:56.579142 24418 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.572824 24496 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.522430 24485 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.554539 24417 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.580134 24418 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.568874 24491 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.551538 24367 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.571641 24320 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.540930 24428 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.579475 24474 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.554390 24430 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.549762 24475 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.553267 24464 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.589394 24411 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.618827 24493 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.619030 24423 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.579962 24466 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.619626 24429 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.619915 24417 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.620642 24462 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:55.912755 23932 rpcz_store.cc:276] 0709 12:12:54.901750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.901940 (+   190us) service_pool.cc:224] Handling call
0709 12:12:54.902309 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.902716 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.902756 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.903296 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.948566 (+ 45270us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:12:54.969373 (+ 20807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.972001 (+  2628us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.892828 (+920827us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b240021ed08. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa54b9c4 00007fb2aa549acb 00007fb2aa704dc9 00007fb2aa704a55 00007fb2aa703fb0 00007fb2ad37a488 00007fb2ad37abe8 00007fb2ad352398 00007fb2ad34d76a 00007fb2ad34cbb6
0709 12:12:55.896757 (+  3929us) raft_consensus.cc:1685] finished
0709 12:12:55.896782 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.912257 (+ 15475us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1141,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":156,"spinlock_wait_cycles":2712832,"thread_start_us":37350,"threads_started":1,"wal-append.queue_time_us":126232}
I20260709 12:12:56.619303 24485 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.545648 23933 rpcz_store.cc:276] 0709 12:12:55.438403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.442880 (+  4477us) service_pool.cc:224] Handling call
0709 12:12:55.854975 (+412095us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:55.855189 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:56.432555 (+577366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.463508 (+ 30953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370240}
I20260709 12:12:56.621127 24428 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.592867 24487 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.629639 24411 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.657588 24308 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.603735 23776 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
W20260709 12:12:56.658706 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 152432us
W20260709 12:12:56.666895 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 166822us
I20260709 12:12:56.594282 24418 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.673333 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:56.674338 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:56.621584 24320 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.655313 24485 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.685247 24490 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.662842 24411 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.687551 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:56.687968 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 161049us
I20260709 12:12:56.663842 24316 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.583876 24496 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.650321 24466 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:56.697777 23860 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:56.654120 24417 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.649734 24474 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.703473 24504 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.684499 24467 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.731702 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 90) took 1526 ms. Trace:
I20260709 12:12:56.694247 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.732180 23681 rpcz_store.cc:276] 0709 12:12:55.217933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.223377 (+  5444us) service_pool.cc:224] Handling call
0709 12:12:56.731628 (+1508251us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:55.747396 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:55.748023 (+   627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:55.748057 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:12:55.748076 (+    19us) tablet.cc:662] Decoding operations
0709 12:12:55.748470 (+   394us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:55.748701 (+   231us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:55.748784 (+    83us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:55.749125 (+   341us) tablet.cc:701] Row locks acquired
0709 12:12:55.749145 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:12:55.749354 (+   209us) write_op.cc:270] Start()
0709 12:12:55.749630 (+   276us) write_op.cc:276] Timestamp: P: 1783599175749290 usec, L: 0
0709 12:12:55.749660 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:12:55.750486 (+   826us) log.cc:844] Serialized 537 byte log entry
0709 12:12:55.872450 (+121964us) op_driver.cc:464] REPLICATION: finished
0709 12:12:56.544254 (+671804us) write_op.cc:301] APPLY: starting
0709 12:12:56.548124 (+  3870us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:56.552983 (+  4859us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:56.558683 (+  5700us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:56.558886 (+   203us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:56.559034 (+   148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:12:56.559076 (+    42us) write_op.cc:312] APPLY: finished
0709 12:12:56.559255 (+   179us) log.cc:844] Serialized 22 byte log entry
0709 12:12:56.559417 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:56.559494 (+    77us) write_op.cc:454] Released schema lock
0709 12:12:56.559528 (+    34us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:56.559861 (+   333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:56.559890 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":650233,"apply.run_cpu_time_us":1846,"apply.run_wall_time_us":24521,"num_ops":1,"prepare.queue_time_us":235431,"prepare.run_cpu_time_us":3399,"prepare.run_wall_time_us":3403,"replication_time_us":122600,"thread_start_us":130198,"threads_started":2}]]}
I20260709 12:12:56.701572 24491 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.687650 24429 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:12:56.663612 24487 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.691892 24485 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.748653 24504 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:56.738667 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:56.748029 24491 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.705816 24490 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.749434 24286 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.732853 24467 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.753190 24327 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.750734 23859 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:56.764248 24491 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.764432 24492 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.768505 24331 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.765661 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:56.767449 23933 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:56.789952 24464 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:56.794415 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 98655.9us
I20260709 12:12:56.775066 24367 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.798576 23899 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:56.789966 24430 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.788780 24488 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.790180 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:56.803481 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:56.571458 23930 rpcz_store.cc:276] 0709 12:12:55.303044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.318063 (+ 15019us) service_pool.cc:224] Handling call
0709 12:12:55.953254 (+635191us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:55.953452 (+   198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:55.953844 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.953880 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:55.954364 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:12:56.474012 (+519648us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:12:56.477004 (+  2992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.506097 (+ 29093us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:56.512947 (+  6850us) raft_consensus.cc:1685] finished
0709 12:12:56.512965 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:56.545132 (+ 32167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12109,"raft.queue_time_us":2587,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":50669,"spinlock_wait_cycles":17469184,"thread_start_us":204746,"threads_started":1,"wal-append.queue_time_us":546087}
I20260709 12:12:56.790479 24409 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.781523 24428 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 12:12:56.804076 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 105500us
I20260709 12:12:56.806994 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:56.794523 24494 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.794445 24502 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:56.801671 24462 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.762161 24499 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:12:56.822567 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.805542 24367 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:56.824110 24308 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.824477 24409 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:56.824990 24320 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.825184 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:56.825542 24417 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.825789 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:56.826066 24510 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.826599 24418 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.801007 24506 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.827171 24475 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.834582 23860 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:56.823352 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.814901 23932 raft_consensus.cc:2368] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:12:56.836388 24487 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.836621 24498 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.084s
I20260709 12:12:56.837159 24485 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.828984 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.834720 24502 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.859222 23781 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:56.837972 23976 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:56.860255 23976 ts_tablet_manager.cc:1403] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 1.047s	user 0.041s	sys 0.068s
I20260709 12:12:56.814551 23928 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:56.807852 24458 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:56.844254 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 142378us
I20260709 12:12:56.841701 24411 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.845065 24508 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:56.799846 24492 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.848899 24320 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.855227 24514 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.857420 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.843449 24462 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.861588 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.862094 24502 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.863547 23975 tablet_bootstrap.cc:492] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:56.871408 23975 ts_tablet_manager.cc:1403] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 1.027s	user 0.014s	sys 0.012s
I20260709 12:12:56.864173 23825 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:56.852276 23778 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.867311 23860 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:56.860383 23776 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:56.866209 24501 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.098s
I20260709 12:12:56.869635 24495 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.845569 24308 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.868225 24320 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.863098 24458 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:56.891490 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:56.864828 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:56.894543 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.839488 24494 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.895174 23860 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:56.873188 24474 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:56.900520 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 225379us
I20260709 12:12:56.897061 24417 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.866731 24430 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:12:56.879293 24331 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.898595 23781 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:56.902002 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:56.903285 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:56.905613 23929 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:56.892822 24458 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.905876 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.904093 23900 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:12:56.900835 24503 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:12:56.870177 24462 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.902030 24517 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:56.892179 24502 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.901067 24467 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.910595 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:56.896425 24418 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:12:56.902818 24320 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.906746 24474 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.863399 24510 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.898444 24496 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.917202 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:56.918015 24515 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.918375 24518 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:56.815937 24504 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.906451 24417 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.864039 24506 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.901294 24490 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:56.910470 23785 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:56.912148 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:56.911975 24331 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:12:56.905965 24498 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:12:56.918579 24502 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.899536 24495 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.931107 24417 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:56.921794 24517 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.012887 24484 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.013713 24486 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:56.927901 24474 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:57.036166 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 133589us
I20260709 12:12:56.874331 24508 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.099535 24496 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.102169 24475 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.102901 24308 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.013192 24507 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.133585 24507 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.878937 24514 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:56.928939 24467 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.028455 24511 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:56.907641 24485 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:12:57.107151 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.101156 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 244) took 1219 ms. Trace:
I20260709 12:12:57.102011 24493 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.110344 24490 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.102039 23776 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:57.110442 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:57.099537 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:57.125123 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:57.133464 24475 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.149654 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:57.132376 24486 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.129151 24520 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.162525 24521 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.159704 23976 raft_consensus.cc:359] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.164330 23975 raft_consensus.cc:359] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.139791 24507 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.142403 24411 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.170552 23976 raft_consensus.cc:385] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.171137 23975 raft_consensus.cc:385] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.131196 24496 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.148612 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.179965 23975 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.180426 23976 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.159298 23857 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.183066 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:57.158896 24511 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.140745 24487 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:12:57.167142 24418 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.156303 24366 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.170306 24486 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.101866 24331 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.132850 24515 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.147266 24493 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.147549 24467 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.172123 24320 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.171963 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.148222 24490 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.185307 24367 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.176491 24506 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.201934 24525 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.197993 23976 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.189760 24511 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.197132 23975 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.190928 24418 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.188392 24499 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.192471 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:57.176458 24527 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:57.194654 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.193861 24492 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.214044 24418 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.173000 24520 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.204303 24320 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:12:57.325800 23975 ts_tablet_manager.cc:1434] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.454s	user 0.010s	sys 0.009s
I20260709 12:12:57.214912 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 242) took 1338 ms. Trace:
I20260709 12:12:57.328194 23976 ts_tablet_manager.cc:1434] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.468s	user 0.014s	sys 0.005s
I20260709 12:12:57.319264 23927 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:57.317695 24510 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.317510 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 245) took 1416 ms. Trace:
I20260709 12:12:57.317585 24464 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.340560 24489 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:57.326678 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.192790 24521 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.321002 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:57.322476 24492 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.310631 24458 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.317780 24522 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.334393 24409 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.329097 24485 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.346346 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.349012 23860 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:57.336052 24518 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.335295 24367 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.212627 24499 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.333811 24494 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.339030 24502 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.352981 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.341082 24489 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.220301 24525 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.368312 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.345763 24411 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.344388 24524 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.314668 24527 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.340595 24464 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.349223 24492 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.141345 23779 rpcz_store.cc:276] 0709 12:12:55.882146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.883396 (+  1250us) service_pool.cc:224] Handling call
0709 12:12:56.142505 (+259109us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:56.583749 (+441244us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68de9c 00007fb2aa68da5b 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:56.588341 (+  4592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:56.646707 (+ 58366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:56.649533 (+  2826us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:56.702666 (+ 53133us) log.cc:844] Serialized 26 byte log entry
0709 12:12:56.875713 (+173047us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:56.888432 (+ 12719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.897339 (+  8907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:57.100914 (+203575us) raft_consensus.cc:1685] finished
0709 12:12:57.100937 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.101064 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2591,"raft.queue_time_us":27321,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":8342,"spinlock_wait_cycles":9859200,"thread_start_us":108736,"threads_started":1,"wal-append.queue_time_us":176570}
I20260709 12:12:57.447204 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:57.361310 24491 consensus_meta.cc:287] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.006s
I20260709 12:12:57.368252 23933 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:57.454066 23860 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:57.360548 24474 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.458392 24487 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.463330 23852 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:57.433413 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.361712 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.353219 24485 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.445650 24521 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.361038 24367 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.422955 24514 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.448628 24484 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.432703 24517 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.363097 24489 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.371868 24494 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.429338 24510 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.450641 24530 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.496574 24525 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.431821 24496 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.458346 24486 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.464511 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.514647 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:57.515543 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:57.546212 24506 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.466200 24320 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:12:57.463961 24466 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.466208 24508 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.459895 24491 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.339357 23784 rpcz_store.cc:276] 0709 12:12:55.900987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.901486 (+   499us) service_pool.cc:224] Handling call
0709 12:12:56.475814 (+574328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:56.532148 (+ 56334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:56.536573 (+  4425us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:56.537456 (+   883us) log.cc:844] Serialized 26 byte log entry
0709 12:12:56.697559 (+160103us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:12:56.701178 (+  3619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.717813 (+ 16635us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:57.218119 (+500306us) raft_consensus.cc:1685] finished
0709 12:12:57.219816 (+  1697us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.315759 (+ 95943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7805,"raft.queue_time_us":75421,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":4229,"thread_start_us":193213,"threads_started":2,"wal-append.queue_time_us":165750}
W20260709 12:12:57.546100 23778 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.001s
I20260709 12:12:57.450114 24411 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.452816 24524 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.514740 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.360986 24522 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.548720 24484 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.329377 23777 rpcz_store.cc:276] 0709 12:12:55.875954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.876651 (+   697us) service_pool.cc:224] Handling call
0709 12:12:56.518909 (+642258us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:56.524439 (+  5530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:56.536056 (+ 11617us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:12:56.729312 (+193256us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.012635 (+283323us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b180016b7c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:12:57.191193 (+178558us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:57.195154 (+  3961us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:57.199320 (+  4166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.204505 (+  5185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.214180 (+  9675us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2930,"prepare.queue_time_us":154884,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":46,"raft.queue_time_us":8733,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6250,"spinlock_wait_cycles":22177408,"thread_start_us":199292,"threads_started":2,"wal-append.queue_time_us":454997}
I20260709 12:12:57.461062 24430 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.550098 24521 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.469681 24487 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:57.577225 23751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.801s (SENT) [suppressed 4 similar messages]
I20260709 12:12:57.548486 24520 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.550976 24490 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.556545 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.559067 24485 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.550994 24494 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.559345 24507 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.616753 23779 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.616343 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.628335 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.616600 23778 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:57.621707 24458 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.562876 24510 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.620428 23852 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
W20260709 12:12:57.640857 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 63680us
I20260709 12:12:57.628789 23856 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:57.620160 24506 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.629922 24464 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.622972 24430 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.630824 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.647716 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.649782 24484 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.643158 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.626555 24521 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.640059 23858 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:57.639917 23851 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:12:57.611788 24530 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.652101 24489 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.768342 23932 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.136260 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 209) took 1255 ms. Trace:
I20260709 12:12:58.136828 23851 rpcz_store.cc:276] 0709 12:12:56.880884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.623088 (+742204us) service_pool.cc:224] Handling call
0709 12:12:58.136182 (+513094us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.138187 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.139266 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 271) took 1037 ms. Trace:
I20260709 12:12:57.992792 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 272) took 1109 ms. Trace:
I20260709 12:12:57.660708 23776 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:57.648288 24492 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.662103 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.717703 24495 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:57.716910 23857 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.522s	user 0.005s	sys 0.005s
I20260709 12:12:58.090914 24502 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:58.121644 24488 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.067647 23680 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:57.818215 23824 leader_election.cc:343] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:58.151134 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 332929us
I20260709 12:12:57.768916 23976 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:58.142009 24489 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.813697 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.141673 24530 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.722569 24506 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.087445 24327 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:57.654090 24527 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.149839 24502 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.998353 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.670657 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.204155 24366 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.204766 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 272) took 1103 ms. Trace:
I20260709 12:12:58.204718 24366 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.672387 24462 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.000386 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.143898 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 258) took 1471 ms. Trace:
I20260709 12:12:58.003821 24498 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.120515 23779 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:57.656104 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.213372 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 237) took 1392 ms. Trace:
I20260709 12:12:58.213640 23933 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:58.144927 24424 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.675477 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:58.214432 23785 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:57.703188 24494 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.074183 24534 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.674662 23854 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:57.652588 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.816787 24493 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:58.221918 23854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 253) took 1551 ms (client timeout 2006 ms). Trace:
I20260709 12:12:58.222254 24494 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.793006 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:57.667578 24491 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.142460 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:58.071292 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:57.767742 24367 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.669145 24522 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.999656 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.005435 23975 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:12:58.227337 23897 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45606 dropped due to backpressure: service queue is full with 50 items
W20260709 12:12:58.177683 23825 leader_election.cc:343] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.759730 24474 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.148996 23751 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.235769 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:12:58.236150 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 87235.4us
W20260709 12:12:58.236654 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59018.5us
I20260709 12:12:58.188638 23929 raft_consensus.cc:2368] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:58.148106 23851 rpcz_store.cc:276] 0709 12:12:57.101409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.137861 (+1036452us) service_pool.cc:224] Handling call
0709 12:12:58.139205 (+  1344us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:57.669376 24417 consensus_meta.cc:287] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.000s
I20260709 12:12:58.179675 23680 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:57.719553 24487 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.666903 24411 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.718336 24524 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.227221 23855 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:57.787102 24430 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.721383 24507 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.824050 24515 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.243763 24506 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:57.798441 24521 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.773553 24499 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.135381 24510 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:57.800060 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:57.775116 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.091377 24466 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.772137 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:57.766309 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.140107 24475 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.214629 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:58.211054 23858 rpcz_store.cc:276] 0709 12:12:56.673117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.686125 (+ 13008us) service_pool.cc:224] Handling call
0709 12:12:57.609096 (+922971us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:12:58.143786 (+534690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110415232}
I20260709 12:12:58.215883 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 245) took 1112 ms. Trace:
I20260709 12:12:58.148985 23778 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:57.819540 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 248) took 1657 ms. Trace:
W20260709 12:12:58.210403 24537 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:12:58.149639 23857 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:58.148276 24467 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:57.790926 23925 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
W20260709 12:12:58.215191 24423 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.023s	sys 0.011s
I20260709 12:12:58.205184 23853 rpcz_store.cc:276] 0709 12:12:57.101651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.670365 (+568714us) service_pool.cc:224] Handling call
0709 12:12:58.204700 (+534335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.199097 24492 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:58.213750 23779 rpcz_store.cc:276] 0709 12:12:56.821177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.464780 (+643603us) service_pool.cc:224] Handling call
0709 12:12:58.213308 (+748528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.205873 24530 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.210731 24429 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:57.673003 24511 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.162374 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.214342 24333 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:58.233642 23927 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:58.214131 23781 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:58.216245 24498 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.219705 24428 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:12:58.167829 24488 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.233812 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 195) took 1402 ms. Trace:
I20260709 12:12:58.217811 24527 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:58.237026 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:12:58.284606 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 47653.2us
I20260709 12:12:58.211094 24327 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.225106 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 270) took 1122 ms. Trace:
I20260709 12:12:58.244408 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 286) took 1055 ms. Trace:
I20260709 12:12:58.244066 23680 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:58.222404 23854 rpcz_store.cc:269] 0709 12:12:56.670601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.672566 (+  1965us) service_pool.cc:224] Handling call
0709 12:12:57.637802 (+965236us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:12:58.221844 (+584042us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111212288}
I20260709 12:12:58.222734 24494 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:57.652648 24418 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.224695 23777 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:58.203657 24286 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:58.217557 24424 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.289364 24527 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.323037 24494 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.323340 24511 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.323659 24494 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.324138 24511 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.324414 24527 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.324584 24511 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.325049 24511 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:12:58.325852 24540 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:12:58.327003 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 273) took 1443 ms. Trace:
I20260709 12:12:58.264149 24466 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.272825 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 197) took 1440 ms. Trace:
I20260709 12:12:58.251500 24506 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.332686 24506 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.250952 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.256702 23785 rpcz_store.cc:276] 0709 12:12:57.103189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.671468 (+568279us) service_pool.cc:224] Handling call
0709 12:12:58.215821 (+544353us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:58.262996 23857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 251) took 1593 ms (client timeout 2012 ms). Trace:
I20260709 12:12:58.258033 23931 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:58.251367 24502 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.251976 24515 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.244767 24474 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.245941 24417 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.263633 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.254480 24510 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:12:58.279500 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 117236us
I20260709 12:12:58.258693 24495 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:58.259817 23859 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
W20260709 12:12:58.269189 24423 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.024s	sys 0.011s
I20260709 12:12:58.268530 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 254) took 1103 ms. Trace:
I20260709 12:12:58.235572 23897 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45606 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 191)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 189)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 190)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 306)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 192)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 186)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201)
I20260709 12:12:58.269675 24487 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.273343 24530 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.352594 24327 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.352855 23784 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:58.353159 24327 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.353835 24530 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.353665 24327 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.354286 24498 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.354399 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 204) took 1497 ms. Trace:
I20260709 12:12:58.258137 23780 rpcz_store.cc:276] 0709 12:12:56.162002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.162405 (+   403us) service_pool.cc:224] Handling call
0709 12:12:56.552331 (+389926us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:56.556520 (+  4189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:56.657005 (+100485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:56.657039 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:56.657565 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:12:56.672081 (+ 14516us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:56.676915 (+  4834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.334997 (+658082us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b18000344e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419
0709 12:12:57.819161 (+484164us) spinlock_profiling.cc:243] Waited 96.3 ms on lock 0x7b5400092c54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:12:57.819214 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:57.819266 (+    52us) raft_consensus.cc:1685] finished
0709 12:12:57.819284 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.819456 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3203,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":21620,"spinlock_wait_cycles":236445696,"thread_start_us":8795,"threads_started":1,"wal-append.queue_time_us":99552,"wal-append.run_cpu_time_us":1644,"wal-append.run_wall_time_us":1033868}
I20260709 12:12:58.149010 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.224655 24534 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.287329 23929 rpcz_store.cc:276] 0709 12:12:57.189359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.813412 (+624053us) service_pool.cc:224] Handling call
0709 12:12:58.244344 (+430932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.341053 24366 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.289433 23680 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.278472 24429 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.286255 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 239) took 1433 ms. Trace:
I20260709 12:12:58.286443 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 284) took 1136 ms. Trace:
I20260709 12:12:58.332871 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.343516 23933 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:58.342181 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.372879 23823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 202) took 1518 ms. Trace:
I20260709 12:12:58.372833 23785 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:58.287984 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:58.273121 24492 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.346206 24510 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.330626 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.350167 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.351926 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 274) took 1135 ms. Trace:
I20260709 12:12:58.344496 23783 raft_consensus.cc:2368] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:58.283499 24333 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.341171 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.283139 23860 rpcz_store.cc:276] 0709 12:12:56.831080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.999390 (+1168310us) service_pool.cc:224] Handling call
0709 12:12:58.233742 (+234352us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:58.359162 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 103628us
I20260709 12:12:58.386804 23900 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:58.387679 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:58.339540 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 241) took 1476 ms. Trace:
I20260709 12:12:58.324143 24494 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.344656 24541 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.383559 23779 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:58.389586 24541 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.390352 24474 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.390957 24494 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:58.338853 23857 rpcz_store.cc:269] 0709 12:12:56.669526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.672478 (+  2952us) service_pool.cc:224] Handling call
0709 12:12:57.043686 (+371208us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:12:58.262924 (+1219238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8685440}
I20260709 12:12:58.391393 24466 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.363852 23776 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:58.361101 23680 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.393313 23680 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.354759 23859 rpcz_store.cc:276] 0709 12:12:56.856501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.766031 (+909530us) service_pool.cc:224] Handling call
0709 12:12:58.354333 (+588302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.283380 24507 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.331822 23856 rpcz_store.cc:276] 0709 12:12:56.831571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.716837 (+885266us) service_pool.cc:224] Handling call
0709 12:12:58.272091 (+555254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.348512 24495 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.366174 24366 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.366510 24530 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.368271 24418 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.369341 24429 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.148263 23930 rpcz_store.cc:276] 0709 12:12:56.882890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.889739 (+  6849us) service_pool.cc:224] Handling call
0709 12:12:57.672636 (+782897us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:57.672795 (+   159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:57.673012 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:12:57.673318 (+   306us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.992079 (+318761us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b18002368c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a
0709 12:12:57.992395 (+   316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:57.992453 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:57.992494 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.992568 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.992709 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12620,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":1769,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"spinlock_wait_cycles":27919488,"thread_start_us":34705,"threads_started":1,"wal-append.queue_time_us":473564}
I20260709 12:12:58.339385 24286 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.287174 23852 rpcz_store.cc:276] 0709 12:12:57.101136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.000126 (+898990us) service_pool.cc:224] Handling call
0709 12:12:58.223714 (+223588us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.373252 23823 rpcz_store.cc:276] 0709 12:12:56.848588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.367285 (+1518697us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:58.385272 24327 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.360832 23932 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:58.352524 24487 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.396170 24507 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.390306 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.348801 23778 rpcz_store.cc:276] 0709 12:12:57.160885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.648179 (+487294us) service_pool.cc:224] Handling call
0709 12:12:58.262774 (+614595us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1924864}
I20260709 12:12:58.392299 23777 rpcz_store.cc:276] 0709 12:12:56.860933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.625366 (+764433us) service_pool.cc:224] Handling call
0709 12:12:58.336564 (+711198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.395263 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:12:58.345584 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 274) took 1459 ms. Trace:
I20260709 12:12:58.456990 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.457322 24487 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.385669 24467 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.394727 24524 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:58.374653 23854 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:58.429806 23976 tablet_bootstrap.cc:654] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.385020 24498 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:58.367519 23782 raft_consensus.cc:2368] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:12:58.371351 23824 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 198)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
I20260709 12:12:58.386157 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.392134 24488 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.428817 24333 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.447604 24366 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.447468 24530 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.428509 24424 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.361929 23781 rpcz_store.cc:276] 0709 12:12:56.850000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.998003 (+1148003us) service_pool.cc:224] Handling call
0709 12:12:58.283088 (+285085us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:58.457386 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 62203.7us
I20260709 12:12:58.446699 24495 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.457573 24491 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:58.458367 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:58.457139 23926 rpcz_store.cc:276] 0709 12:12:56.884266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.889695 (+  5429us) service_pool.cc:224] Handling call
0709 12:12:56.909219 (+ 19524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:57.149750 (+240531us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:12:57.801712 (+651962us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b1800236920. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf
0709 12:12:57.801885 (+   173us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.202298 (+400413us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.202357 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:12:58.202401 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.288486 (+ 86085us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002440d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:12:58.336472 (+ 47986us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.343451 (+  6979us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":287809,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1708,"raft.queue_time_us":108,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":555890944,"thread_start_us":657997,"threads_started":2,"wal-append.queue_time_us":400335}
I20260709 12:12:58.327409 23934 rpcz_store.cc:276] 0709 12:12:56.883350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.889564 (+  6214us) service_pool.cc:224] Handling call
0709 12:12:57.673461 (+783897us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:57.673702 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:57.674065 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:57.674096 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:57.717087 (+ 42991us) log.cc:844] Serialized 26 byte log entry
0709 12:12:57.995517 (+278430us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b18002369e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:12:58.236951 (+241434us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:58.243684 (+  6733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.246498 (+  2814us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.326718 (+ 80220us) raft_consensus.cc:1685] finished
0709 12:12:58.326751 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.326929 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42561,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":37228,"spinlock_wait_cycles":23946880,"thread_start_us":230563,"threads_started":1,"wal-append.queue_time_us":531454}
I20260709 12:12:58.512702 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.395191 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 275) took 1178 ms. Trace:
I20260709 12:12:58.510102 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.514477 23858 rpcz_store.cc:276] 0709 12:12:57.217020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.267828 (+1050808us) service_pool.cc:224] Handling call
0709 12:12:58.392984 (+125156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.377753 23851 rpcz_store.cc:276] 0709 12:12:57.216726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.246818 (+1030092us) service_pool.cc:224] Handling call
0709 12:12:58.351838 (+105020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.514981 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.363281 23927 rpcz_store.cc:276] 0709 12:12:57.147437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.776498 (+629061us) service_pool.cc:224] Handling call
0709 12:12:58.283240 (+506742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.394577 23853 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:58.515533 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.516016 23776 raft_consensus.cc:2410] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:58.516197 23781 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:58.511682 24286 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.516999 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.517161 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.450008 23749 leader_election.cc:400] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:12:58.517750 23749 leader_election.cc:403] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:12:58.518325 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 86744.1us
I20260709 12:12:58.519209 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:58.456079 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 270) took 1574 ms. Trace:
W20260709 12:12:58.520457 23749 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.413132 24411 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:58.509153 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:58.457007 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:58.521566 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.394162 24541 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.517288 23777 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.531647 24367 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:58.511114 24366 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.524627 24286 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.535068 24286 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.488015 24507 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.497825 24524 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.528107 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.547031 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.521838 23899 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.534314 24424 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:58.547851 24424 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.548345 24423 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.548447 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.548763 24423 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.549081 24333 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.549162 24423 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.549612 24333 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.550028 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.550696 23931 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:58.550815 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.538129 23975 tablet_bootstrap.cc:654] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.551625 24428 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.552021 24428 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.507390 24498 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:58.536768 24286 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.544044 24510 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.553938 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.553476 24428 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.556471 23931 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:58.509393 24534 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.527407 24411 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:12:58.545258 23925 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 184) took 1630 ms (client timeout 1630 ms). Trace:
I20260709 12:12:58.546021 23928 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:58.550203 24493 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:58.550004 24333 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.567795 24411 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.503121 23855 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:58.571048 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 267) took 1470 ms. Trace:
I20260709 12:12:58.548243 24424 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:12:58.552714 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58239.9us
I20260709 12:12:58.552663 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.510051 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.500978 24467 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.554423 23899 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.528508 24418 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:12:58.512005 24530 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.517839 23781 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:58.580313 23899 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:58.517738 23776 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:58.506495 23780 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:58.513548 24491 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.557337 24367 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:58.544775 24417 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.557554 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.601917 24367 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.602425 24367 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.603055 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:58.604941 23782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 228) took 1786 ms (client timeout 1859 ms). Trace:
I20260709 12:12:58.532545 23779 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:58.563807 24524 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.574884 24515 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:12:58.575229 23825 leader_election.cc:343] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:58.565887 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 58106.1us
I20260709 12:12:58.571236 24493 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:58.582175 24489 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:58.571425 23855 rpcz_store.cc:276] 0709 12:12:57.100254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.792718 (+692464us) service_pool.cc:224] Handling call
0709 12:12:58.570974 (+778256us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:58.566599 23925 rpcz_store.cc:269] 0709 12:12:56.922845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.176808 (+253963us) service_pool.cc:224] Handling call
0709 12:12:58.545029 (+1368221us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54002a4754. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:58.545159 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323574144}
I20260709 12:12:58.590649 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.590561 24423 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.520031 23932 rpcz_store.cc:276] 0709 12:12:56.881702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.891356 (+  9654us) service_pool.cc:224] Handling call
0709 12:12:57.768038 (+876682us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:57.768221 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.455763 (+687542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.456012 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9355648}
I20260709 12:12:58.592927 24511 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.539912 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 252)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289)
I20260709 12:12:58.582907 24462 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:58.842940 24462 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.843369 24462 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:58.844179 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.845012 23929 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:58.582461 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:58.844390 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43422 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 252)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 292)
W20260709 12:12:58.847857 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 265445us
I20260709 12:12:58.847958 24466 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:58.848369 24466 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.620234 24493 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.848788 24466 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:58.617254 24489 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:58.849503 24489 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.850772 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:58.854692 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 288) took 1636 ms (client timeout 1982 ms). Trace:
W20260709 12:12:58.856251 23929 rpcz_store.cc:269] 0709 12:12:57.217925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.843911 (+1625986us) service_pool.cc:224] Handling call
0709 12:12:58.854631 (+ 10720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.577666 24495 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.617405 24498 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.862084 24495 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.595803 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:58.605397 23782 rpcz_store.cc:269] 0709 12:12:56.818852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.132490 (+313638us) service_pool.cc:224] Handling call
0709 12:12:58.604666 (+1472176us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540010a754. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:58.604841 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2658139520}
I20260709 12:12:59.422602 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.423240 23782 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:58.602274 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.424960 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 312) took 1608 ms (client timeout 1768 ms). Trace:
W20260709 12:12:59.425328 23931 rpcz_store.cc:269] 0709 12:12:57.816467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.601991 (+785524us) service_pool.cc:224] Handling call
0709 12:12:59.424894 (+822903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.426445 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.427212 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 313) took 1610 ms (client timeout 1954 ms). Trace:
I20260709 12:12:58.644788 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.644790 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.648538 23824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 198) took 1816 ms. Trace:
I20260709 12:12:59.442764 23824 rpcz_store.cc:276] 0709 12:12:56.831828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.648489 (+1816661us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:59.442869 23925 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
W20260709 12:12:58.810092 23783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 257) took 1648 ms (client timeout 1627 ms). Trace:
I20260709 12:12:58.818341 23777 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:58.622701 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:58.858331 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 240) took 2004 ms (client timeout 2135 ms). Trace:
I20260709 12:12:58.863221 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.443930 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 257) took 1071 ms. Trace:
W20260709 12:12:59.443470 23824 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items [suppressed 6 similar messages]
I20260709 12:12:58.864466 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.848409 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43422 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 252)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 292)
I20260709 12:12:59.444420 23777 rpcz_store.cc:276] 0709 12:12:58.372300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.458083 (+ 85783us) service_pool.cc:224] Handling call
0709 12:12:59.443868 (+985785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.445011 23855 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:59.445514 23855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 220) took 1803 ms (client timeout 2020 ms). Trace:
I20260709 12:12:59.445472 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.444590 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 305) took 1630 ms (client timeout 1952 ms). Trace:
I20260709 12:12:59.446237 23777 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:58.449080 23823 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 198)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
I20260709 12:12:59.446822 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 260) took 1062 ms. Trace:
W20260709 12:12:59.447005 23933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 190) took 1827 ms (client timeout 1884 ms). Trace:
W20260709 12:12:59.447428 23933 rpcz_store.cc:269] 0709 12:12:57.626444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.864135 (+1237691us) service_pool.cc:224] Handling call
0709 12:12:59.446945 (+582810us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.448419 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.915441 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.449170 23933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 192) took 1797 ms (client timeout 2126 ms). Trace:
I20260709 12:12:59.449417 23934 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:59.443732 23783 rpcz_store.cc:269] 0709 12:12:57.161702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.313099 (+151397us) service_pool.cc:224] Handling call
0709 12:12:58.809854 (+1496755us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b540035acd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:58.810004 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263859456}
I20260709 12:12:59.450727 23900 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.450851 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.451538 23783 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:59.452092 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 262) took 1055 ms. Trace:
I20260709 12:12:59.452857 24543 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.917294 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:58.621260 24286 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.677346 24428 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.444520 23824 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
I20260709 12:12:58.658453 24367 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.452548 23900 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:59.459985 24367 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.460547 24367 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.857908 24462 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.446487 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43422 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 292)
I20260709 12:12:58.598079 24492 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.607286 24506 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.681768 24511 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.614357 24515 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.677343 24467 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.927609 24417 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.695380 23975 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:12:59.447184 23777 rpcz_store.cc:276] 0709 12:12:58.384606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.445211 (+1060605us) service_pool.cc:224] Handling call
0709 12:12:59.446764 (+  1553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.470213 23975 ts_tablet_manager.cc:1403] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 1.465s	user 0.020s	sys 0.023s
I20260709 12:12:58.782456 23976 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:12:59.472399 23976 ts_tablet_manager.cc:1403] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 1.704s	user 0.038s	sys 0.014s
I20260709 12:12:58.687091 24333 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.477679 24333 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.478227 24333 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.478658 24333 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.646543 24423 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.681478 24424 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.482468 24424 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:58.659660 24507 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.697206 24524 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:58.599112 24491 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:58.608671 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 230) took 1779 ms. Trace:
I20260709 12:12:58.936076 24498 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:12:58.858445 24489 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.345659 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:58.618821 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 232) took 1798 ms. Trace:
I20260709 12:12:58.645393 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.491716 23785 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:59.492234 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 254) took 1129 ms. Trace:
I20260709 12:12:59.379881 24521 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:59.492712 23785 rpcz_store.cc:276] 0709 12:12:58.362736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.617878 (+255142us) service_pool.cc:224] Handling call
0709 12:12:59.492174 (+874296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.493265 24524 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.493147 24521 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.493595 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.494136 23785 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.494558 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246) took 1154 ms. Trace:
I20260709 12:12:59.494829 23785 rpcz_store.cc:276] 0709 12:12:58.339767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.493346 (+1153579us) service_pool.cc:224] Handling call
0709 12:12:59.494508 (+  1162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.495174 24521 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.381302 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 271) took 2499 ms. Trace:
W20260709 12:12:59.491206 23858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 218) took 2024 ms (client timeout 2141 ms). Trace:
I20260709 12:12:58.248188 23898 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45606 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 191)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 189)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 190)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 306)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 187)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 192)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 186)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201)
W20260709 12:12:59.496062 23858 rpcz_store.cc:269] 0709 12:12:57.467037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.345313 (+1878276us) service_pool.cc:224] Handling call
0709 12:12:59.491140 (+145827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.497186 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.497520 24507 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:59.497953 23858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225) took 1692 ms (client timeout 1901 ms). Trace:
W20260709 12:12:59.498332 23858 rpcz_store.cc:269] 0709 12:12:57.805124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.496894 (+1691770us) service_pool.cc:224] Handling call
0709 12:12:59.497900 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.498790 24489 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.405532 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.407584 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.499887 23852 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:59.500136 23857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 216) took 2039 ms (client timeout 2041 ms). Trace:
W20260709 12:12:59.500398 23852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 219) took 2032 ms (client timeout 2129 ms). Trace:
W20260709 12:12:59.409780 23899 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.000s	sys 0.028s
I20260709 12:12:58.645547 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.414048 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.416288 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.417151 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.418057 23926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 191) took 1786 ms (client timeout 1583 ms). Trace:
I20260709 12:12:59.420480 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.503326 23899 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.000s	sys 0.028s
W20260709 12:12:59.503715 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 825692us
I20260709 12:12:58.595611 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 215) took 1135 ms. Trace:
I20260709 12:12:59.495719 23928 rpcz_store.cc:276] 0709 12:12:56.882115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.889860 (+  7745us) service_pool.cc:224] Handling call
0709 12:12:57.463202 (+573342us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:12:58.545648 (+1082446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.569497 (+ 23849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.381054 (+811557us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002a4750. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.381223 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2496245376}
W20260709 12:12:59.504976 23926 rpcz_store.cc:269] 0709 12:12:57.644047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.417833 (+1773786us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:59.417973 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:59.503635 23851 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:59.506645 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 211) took 2047 ms (client timeout 2126 ms). Trace:
W20260709 12:12:59.507038 23851 rpcz_store.cc:269] 0709 12:12:57.459365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.416866 (+1957501us) service_pool.cc:224] Handling call
0709 12:12:59.506586 (+ 89720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.499073 23898 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45620 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 186)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 317)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 206)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 316)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 324)
I20260709 12:12:59.508071 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.508718 23851 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.508791 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.504666 23781 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
W20260709 12:12:59.509258 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285) took 1240 ms (client timeout 1533 ms). Trace:
I20260709 12:12:59.509673 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 268) took 1081 ms. Trace:
W20260709 12:12:59.509763 23928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 310) took 1693 ms (client timeout 2143 ms). Trace:
I20260709 12:12:59.510166 23781 rpcz_store.cc:276] 0709 12:12:58.428684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.621567 (+192883us) service_pool.cc:224] Handling call
0709 12:12:59.509616 (+888049us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.510546 23928 rpcz_store.cc:269] 0709 12:12:57.815972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.508430 (+1692458us) service_pool.cc:224] Handling call
0709 12:12:59.509701 (+  1271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.510947 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.511225 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.511870 23781 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.512673 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.512375 23681 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:59.509641 23851 rpcz_store.cc:269] 0709 12:12:58.269008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.507738 (+1238730us) service_pool.cc:224] Handling call
0709 12:12:59.509202 (+  1464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.505264 23856 rpcz_store.cc:276] 0709 12:12:57.460447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.461632 (+1001185us) service_pool.cc:224] Handling call
0709 12:12:58.595516 (+133884us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.514186 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.441563 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 96) took 3881 ms. Trace:
I20260709 12:12:58.899649 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.453737 24493 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.454285 24543 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:59.443960 23784 rpcz_store.cc:269] 0709 12:12:56.853988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.624618 (+770630us) service_pool.cc:224] Handling call
0709 12:12:57.771935 (+147317us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:12:58.858136 (+1086201us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b5400167e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:58.858268 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225584128}
I20260709 12:12:59.515115 23928 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.515391 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.446549 23929 rpcz_store.cc:269] 0709 12:12:57.814582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.862937 (+1048355us) service_pool.cc:224] Handling call
0709 12:12:59.444522 (+581585us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.516068 23928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 309) took 1700 ms (client timeout 2157 ms). Trace:
W20260709 12:12:59.445988 23855 rpcz_store.cc:269] 0709 12:12:57.642377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.622372 (+979995us) service_pool.cc:224] Handling call
0709 12:12:59.445457 (+823085us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.516812 23928 rpcz_store.cc:269] 0709 12:12:57.815687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.513767 (+1698080us) service_pool.cc:224] Handling call
0709 12:12:59.516008 (+  2241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.517510 24493 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.517943 24493 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.518381 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.514937 23680 rpcz_store.cc:276] 0709 12:12:55.560190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.560352 (+   162us) service_pool.cc:224] Handling call
0709 12:12:59.441477 (+3881125us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:58.898154 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:58.898534 (+   380us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:58.898556 (+    22us) write_op.cc:435] Acquired schema lock
0709 12:12:58.898567 (+    11us) tablet.cc:662] Decoding operations
0709 12:12:58.899214 (+   647us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:58.899269 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:58.899288 (+    19us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:12:58.899648 (+   360us) tablet.cc:701] Row locks acquired
0709 12:12:58.899666 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:12:58.899774 (+   108us) write_op.cc:270] Start()
0709 12:12:58.899860 (+    86us) write_op.cc:276] Timestamp: P: 1783599178899741 usec, L: 0
0709 12:12:58.899874 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 12:12:58.900433 (+   559us) log.cc:844] Serialized 2463 byte log entry
0709 12:12:58.903943 (+  3510us) op_driver.cc:464] REPLICATION: finished
0709 12:12:58.905368 (+  1425us) write_op.cc:301] APPLY: starting
0709 12:12:58.905425 (+    57us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:58.905545 (+   120us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:58.905640 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:58.906187 (+   547us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:58.906328 (+   141us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:12:58.906356 (+    28us) write_op.cc:312] APPLY: finished
0709 12:12:58.906543 (+   187us) log.cc:844] Serialized 52 byte log entry
0709 12:12:58.906689 (+   146us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:58.906786 (+    97us) write_op.cc:454] Released schema lock
0709 12:12:58.906827 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:58.907948 (+  1121us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:58.907975 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1026,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":1236,"apply.run_cpu_time_us":1736,"apply.run_wall_time_us":2557,"num_ops":6,"prepare.queue_time_us":1941,"prepare.run_cpu_time_us":3333,"prepare.run_wall_time_us":3333,"raft.queue_time_us":878,"raft.run_cpu_time_us":1078,"raft.run_wall_time_us":1078,"replication_time_us":3957,"thread_start_us":3439,"threads_started":4,"wal-append.queue_time_us":975}]]}
I20260709 12:12:59.519229 23928 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:59.519886 23928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 196) took 1583 ms (client timeout 2065 ms). Trace:
W20260709 12:12:59.520256 23928 rpcz_store.cc:269] 0709 12:12:58.000710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.518084 (+1517374us) service_pool.cc:224] Handling call
0709 12:12:59.519825 (+  1741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.521235 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.522050 23928 raft_consensus.cc:2393] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.522514 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 198) took 1521 ms. Trace:
I20260709 12:12:59.522878 23928 rpcz_store.cc:276] 0709 12:12:58.001402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.520965 (+1519563us) service_pool.cc:224] Handling call
0709 12:12:59.522456 (+  1491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.515407 23853 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:12:59.523887 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.453476 23930 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:12:59.524624 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 294) took 1974 ms (client timeout 1715 ms). Trace:
I20260709 12:12:59.514991 24491 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:59.525660 23930 rpcz_store.cc:269] 0709 12:12:57.550202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.916992 (+1366790us) service_pool.cc:224] Handling call
0709 12:12:59.524546 (+607554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.526374 23932 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.526646 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.516615 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.527199 24491 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:59.527407 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 186) took 2165 ms (client timeout 2737 ms). Trace:
I20260709 12:12:59.516096 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.527828 23932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 293) took 2059 ms (client timeout 2226 ms). Trace:
W20260709 12:12:59.527844 23930 rpcz_store.cc:269] 0709 12:12:57.413649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.526381 (+2112732us) service_pool.cc:224] Handling call
0709 12:12:59.527350 (+   969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.517231 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.528304 23932 rpcz_store.cc:269] 0709 12:12:57.468666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.644343 (+1175677us) service_pool.cc:224] Handling call
0709 12:12:59.527636 (+883293us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b54000f8cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.527777 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224908416}
I20260709 12:12:59.529207 23855 raft_consensus.cc:2410] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:59.529596 23928 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:59.463505 24462 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:58.857044 24466 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:12:59.449946 23934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 292) took 2001 ms (client timeout 2063 ms). Trace:
W20260709 12:12:59.530926 23934 rpcz_store.cc:269] 0709 12:12:57.448156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.915074 (+1466918us) service_pool.cc:224] Handling call
0709 12:12:59.449889 (+534815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.531924 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.464486 23927 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.528316 23784 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.532212 24466 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.466594 24506 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.467276 24286 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.457906 24428 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.467824 24515 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.544816 24493 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.547437 24493 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:59.459038 23933 rpcz_store.cc:269] 0709 12:12:57.652023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.448135 (+1796112us) service_pool.cc:224] Handling call
0709 12:12:59.449117 (+   982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.452452 23783 rpcz_store.cc:276] 0709 12:12:58.396211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.450597 (+1054386us) service_pool.cc:224] Handling call
0709 12:12:59.452013 (+  1416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.552333 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.552409 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.457947 23824 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
I20260709 12:12:59.553098 23933 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:59.553705 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 202) took 1551 ms. Trace:
I20260709 12:12:59.554045 23933 rpcz_store.cc:276] 0709 12:12:58.002514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.551980 (+1549466us) service_pool.cc:224] Handling call
0709 12:12:59.553643 (+  1663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.555047 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.466046 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43422 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 294)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 292)
I20260709 12:12:59.556468 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 315) took 1368 ms. Trace:
I20260709 12:12:59.556869 23933 rpcz_store.cc:276] 0709 12:12:58.188014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.554743 (+1366729us) service_pool.cc:224] Handling call
0709 12:12:59.556396 (+  1653us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.558445 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.483966 23859 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.559244 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 200) took 1557 ms. Trace:
W20260709 12:12:59.560635 23859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 269) took 2459 ms (client timeout 2363 ms). Trace:
I20260709 12:12:59.484833 24417 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.560613 24515 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.488690 23780 rpcz_store.cc:276] 0709 12:12:56.819335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.372078 (+1552743us) service_pool.cc:224] Handling call
0709 12:12:58.598832 (+226754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.561398 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.563035 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.563757 23780 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.564245 23899 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.564731 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.565375 23780 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:59.561086 23859 rpcz_store.cc:269] 0709 12:12:57.100861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.590344 (+1489483us) service_pool.cc:224] Handling call
0709 12:12:59.560424 (+970080us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54002dfd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.560565 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1977162112}
I20260709 12:12:59.566207 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.566651 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.566874 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 288) took 1035 ms. Trace:
I20260709 12:12:59.489341 24333 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.490113 23975 raft_consensus.cc:359] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.567830 23975 raft_consensus.cc:385] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:59.491122 24498 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.492211 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.567961 24462 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.384979 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.407423 24495 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.568374 24462 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:59.568799 24462 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.569972 24498 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.570607 23778 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
W20260709 12:12:59.571110 23778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 247) took 1231 ms (client timeout 1640 ms). Trace:
I20260709 12:12:59.568384 23777 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.573277 24417 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.574229 23975 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:12:59.574980 23975 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.577204 23975 ts_tablet_manager.cc:1434] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.107s	user 0.006s	sys 0.000s
I20260709 12:12:59.501247 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 104) took 3610 ms. Trace:
W20260709 12:12:59.501718 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49237.3us
I20260709 12:12:59.491326 23779 rpcz_store.cc:276] 0709 12:12:56.819906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.456724 (+1636818us) service_pool.cc:224] Handling call
0709 12:12:58.617789 (+161065us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27264}
I20260709 12:12:59.499269 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.578258 23683 rpcz_store.cc:276] 0709 12:12:55.890923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.891059 (+   136us) service_pool.cc:224] Handling call
0709 12:12:59.501196 (+3610137us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
W20260709 12:12:59.578840 23858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226) took 1362 ms (client timeout 1558 ms). Trace:
I20260709 12:12:59.502269 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.571461 23778 rpcz_store.cc:269] 0709 12:12:58.339955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.456637 (+116682us) service_pool.cc:224] Handling call
0709 12:12:59.384811 (+928174us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68de9c 00007fb2aa68da5b 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:12:59.571052 (+186241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4388736}
I20260709 12:12:59.579725 23785 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.580183 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 281) took 1051 ms. Trace:
I20260709 12:12:59.580487 23785 rpcz_store.cc:276] 0709 12:12:58.529079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.495460 (+966381us) service_pool.cc:224] Handling call
0709 12:12:59.580130 (+ 84670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.580936 23783 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.581449 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.582015 23785 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.503962 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 252) took 1155 ms. Trace:
I20260709 12:12:59.584306 23776 rpcz_store.cc:276] 0709 12:12:58.348313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.420214 (+1071901us) service_pool.cc:224] Handling call
0709 12:12:59.503901 (+ 83687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.585361 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.586616 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:12:59.504102 23860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 223) took 1699 ms (client timeout 1656 ms). Trace:
W20260709 12:12:59.587741 23860 rpcz_store.cc:269] 0709 12:12:57.804614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.416029 (+1611415us) service_pool.cc:224] Handling call
0709 12:12:59.504037 (+ 88008us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.500787 23852 rpcz_store.cc:269] 0709 12:12:57.467502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.405223 (+1937721us) service_pool.cc:224] Handling call
0709 12:12:59.500343 (+ 95120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.580274 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.500528 23857 rpcz_store.cc:269] 0709 12:12:57.460686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.407278 (+1946592us) service_pool.cc:224] Handling call
0709 12:12:59.500074 (+ 92796us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.504745 23854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 210) took 2045 ms (client timeout 2166 ms). Trace:
I20260709 12:12:59.589399 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 286) took 1059 ms. Trace:
W20260709 12:12:59.589630 23854 rpcz_store.cc:269] 0709 12:12:57.459039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.413715 (+1954676us) service_pool.cc:224] Handling call
0709 12:12:59.504678 (+ 90963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.509258 23898 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45606 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 297)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 196)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 198)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 186)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 203)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 185)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 202)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 200)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 317)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 206)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 316)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 324)
I20260709 12:12:59.580575 24495 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.514241 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.592054 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278) took 1428 ms (client timeout 1640 ms). Trace:
W20260709 12:12:59.592448 23851 rpcz_store.cc:269] 0709 12:12:58.164110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.513988 (+1349878us) service_pool.cc:224] Handling call
0709 12:12:59.591993 (+ 78005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.516208 23781 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.593456 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 272) took 1163 ms. Trace:
I20260709 12:12:59.593791 23781 rpcz_store.cc:276] 0709 12:12:58.429691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.513388 (+1083697us) service_pool.cc:224] Handling call
0709 12:12:59.593383 (+ 79995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.594707 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.524041 23853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 203) took 2670 ms (client timeout 2737 ms). Trace:
I20260709 12:12:59.596124 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.596865 23779 raft_consensus.cc:2368] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:59.597707 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.598199 23779 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:59.529052 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.601163 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 203) took 1598 ms (client timeout 2129 ms). Trace:
W20260709 12:12:59.601540 23930 rpcz_store.cc:269] 0709 12:12:58.002784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.528801 (+1526017us) service_pool.cc:224] Handling call
0709 12:12:59.601099 (+ 72298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.529676 23855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217) took 2065 ms (client timeout 2424 ms). Trace:
I20260709 12:12:59.457976 23823 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 285)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 278)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
W20260709 12:12:59.602427 23855 rpcz_store.cc:269] 0709 12:12:57.464426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.517002 (+2052576us) service_pool.cc:224] Handling call
0709 12:12:59.529620 (+ 12618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.527693 23929 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:59.603124 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 297) took 1793 ms (client timeout 2184 ms). Trace:
W20260709 12:12:59.603447 23929 rpcz_store.cc:269] 0709 12:12:57.809913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.516408 (+1706495us) service_pool.cc:224] Handling call
0709 12:12:59.603068 (+ 86660us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.534112 23927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 291) took 2120 ms (client timeout 1761 ms). Trace:
I20260709 12:12:59.587499 23776 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:12:59.604481 23927 rpcz_store.cc:269] 0709 12:12:57.414083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.850404 (+1436321us) service_pool.cc:224] Handling call
0709 12:12:59.533913 (+683509us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54000e3654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.534045 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409236352}
I20260709 12:12:59.606190 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.606978 23776 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.607376 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 266) took 1179 ms. Trace:
I20260709 12:12:59.607712 23776 rpcz_store.cc:276] 0709 12:12:58.428144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.605907 (+1177763us) service_pool.cc:224] Handling call
0709 12:12:59.607320 (+  1413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.608250 24495 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.595373 23781 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:12:59.608610 23751 leader_election.cc:336] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.801s (SENT)
I20260709 12:12:59.609467 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.610139 23781 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:12:59.611069 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:12:59.595736 23853 rpcz_store.cc:269] 0709 12:12:56.853668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.899267 (+2045599us) service_pool.cc:224] Handling call
0709 12:12:59.523993 (+624726us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.579217 23858 rpcz_store.cc:269] 0709 12:12:58.216312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.498999 (+1282687us) service_pool.cc:224] Handling call
0709 12:12:59.578777 (+ 79778us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.615083 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.615769 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.616417 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.616663 23857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288) took 1274 ms (client timeout 1668 ms). Trace:
W20260709 12:12:59.617214 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227) took 1398 ms (client timeout 1801 ms). Trace:
W20260709 12:12:59.464721 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:59.526589 24543 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:59.443467 23925 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 187) took 2006 ms (client timeout 2201 ms). Trace:
I20260709 12:12:59.514319 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.618023 23851 rpcz_store.cc:269] 0709 12:12:58.219199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.616125 (+1396926us) service_pool.cc:224] Handling call
0709 12:12:59.617155 (+  1030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.618449 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.618549 23856 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:12:59.619140 23856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 221) took 1976 ms (client timeout 2188 ms). Trace:
I20260709 12:12:59.541682 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.430014 23931 rpcz_store.cc:269] 0709 12:12:57.816754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.426180 (+1609426us) service_pool.cc:224] Handling call
0709 12:12:59.427159 (+   979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.543450 23681 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:59.431113 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.532968 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 263) took 1105 ms. Trace:
I20260709 12:12:59.475101 23976 raft_consensus.cc:359] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.559621 23933 rpcz_store.cc:276] 0709 12:12:58.001981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.558134 (+1556153us) service_pool.cc:224] Handling call
0709 12:12:59.559188 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.617076 23857 rpcz_store.cc:269] 0709 12:12:58.342128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.615389 (+1273261us) service_pool.cc:224] Handling call
0709 12:12:59.616603 (+  1214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.540203 23934 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.637287 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:59.646274 23932 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:12:59.646149 23934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 204) took 1643 ms (client timeout 2141 ms). Trace:
I20260709 12:12:59.646872 24543 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.637154 23975 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:12:59.567502 23859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 280) took 1402 ms (client timeout 1798 ms). Trace:
W20260709 12:12:59.647166 23934 rpcz_store.cc:269] 0709 12:12:58.003053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.531668 (+1528615us) service_pool.cc:224] Handling call
0709 12:12:59.646055 (+114387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.619199 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.648761 23860 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.649042 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.647692 24543 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.626061 23855 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.649677 24543 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:59.650086 23855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205) took 2784 ms (client timeout 3143 ms). Trace:
W20260709 12:12:59.650476 23855 rpcz_store.cc:269] 0709 12:12:56.865310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.614754 (+2749444us) service_pool.cc:224] Handling call
0709 12:12:59.650027 (+ 35273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.651072 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.650085 23782 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.651867 23931 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.652064 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 290) took 1119 ms. Trace:
I20260709 12:12:59.652347 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195) took 1982 ms. Trace:
I20260709 12:12:59.651415 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.619287 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231) took 1285 ms. Trace:
I20260709 12:12:59.646785 23778 rpcz_store.cc:276] 0709 12:12:58.530448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.579957 (+1049509us) service_pool.cc:224] Handling call
0709 12:12:59.589337 (+  9380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.653187 23854 rpcz_store.cc:276] 0709 12:12:58.333622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.618177 (+1284555us) service_pool.cc:224] Handling call
0709 12:12:59.619230 (+  1053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.653877 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.654225 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.654822 23854 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.654964 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.655324 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232) took 1284 ms. Trace:
I20260709 12:12:59.646579 24493 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.652420 23782 rpcz_store.cc:276] 0709 12:12:58.532846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.430866 (+898020us) service_pool.cc:224] Handling call
0709 12:12:59.652001 (+221135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.664615 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.665227 24493 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:12:59.665450 23853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283) took 1437 ms (client timeout 1847 ms). Trace:
I20260709 12:12:59.665673 24493 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.665810 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.666635 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287) took 1326 ms. Trace:
I20260709 12:12:59.666955 23858 rpcz_store.cc:276] 0709 12:12:58.340242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.665555 (+1325313us) service_pool.cc:224] Handling call
0709 12:12:59.666579 (+  1024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.667256 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.647383 23784 rpcz_store.cc:276] 0709 12:12:58.427272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.515853 (+1088581us) service_pool.cc:224] Handling call
0709 12:12:59.532909 (+ 17056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.658437 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.668059 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.668216 23852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284) took 1418 ms (client timeout 1832 ms). Trace:
I20260709 12:12:59.668406 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.658636 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.668994 23784 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.669713 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.669929 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.670696 23927 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.670908 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.671077 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197) took 1670 ms. Trace:
W20260709 12:12:59.646960 23932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 185) took 2714 ms (client timeout 3211 ms). Trace:
W20260709 12:12:59.665843 23853 rpcz_store.cc:269] 0709 12:12:58.228154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.664288 (+1436134us) service_pool.cc:224] Handling call
0709 12:12:59.665394 (+  1106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.671734 23932 rpcz_store.cc:269] 0709 12:12:56.958000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.529598 (+2571598us) service_pool.cc:224] Handling call
0709 12:12:59.646894 (+117296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.672432 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.672691 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.673231 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230) took 1343 ms. Trace:
W20260709 12:12:59.648132 23859 rpcz_store.cc:269] 0709 12:12:58.164646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.566379 (+1401733us) service_pool.cc:224] Handling call
0709 12:12:59.567444 (+  1065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.652735 23931 rpcz_store.cc:276] 0709 12:12:57.670219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.650785 (+1980566us) service_pool.cc:224] Handling call
0709 12:12:59.652212 (+  1427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.673615 23853 rpcz_store.cc:276] 0709 12:12:58.329583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.672149 (+1342566us) service_pool.cc:224] Handling call
0709 12:12:59.673167 (+  1018us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.674377 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.647204 23976 raft_consensus.cc:385] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:59.652781 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.674469 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.647133 23856 rpcz_store.cc:269] 0709 12:12:57.642685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.514075 (+1871390us) service_pool.cc:224] Handling call
0709 12:12:59.619085 (+105010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.659050 23854 rpcz_store.cc:276] 0709 12:12:58.370864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.653866 (+1283002us) service_pool.cc:224] Handling call
0709 12:12:59.655266 (+  1400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.675171 23929 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:59.675688 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.675776 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.608644 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.676100 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.594208 23898 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45606 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 317)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 195)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 197)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 211)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 208)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 207)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 224)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 210)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 218)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 217)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 206)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 228)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 223)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 316)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 324)
I20260709 12:12:59.677150 23976 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:12:59.635023 23779 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:12:59.659539 23785 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:59.657136 23823 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 299)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 309)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 300)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 247)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 245)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 307)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 302)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306)
I20260709 12:12:59.587201 23824 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 205)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 288)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 231)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 283)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 284)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 287)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 232)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 230)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 247)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 245)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
I20260709 12:12:59.622547 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.672425 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:59.538188 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 314) took 1351 ms. Trace:
I20260709 12:12:59.567263 23780 rpcz_store.cc:276] 0709 12:12:58.531040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.565965 (+1034925us) service_pool.cc:224] Handling call
0709 12:12:59.566817 (+   852us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.649209 23860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 201) took 2805 ms (client timeout 3211 ms). Trace:
W20260709 12:12:59.618150 23925 rpcz_store.cc:269] 0709 12:12:57.458583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.644463 (+1185880us) service_pool.cc:224] Handling call
0709 12:12:59.443407 (+798944us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24320}
W20260709 12:12:59.679327 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56865.4us
I20260709 12:12:59.703918 23784 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:12:59.678651 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.674628 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.698634 23781 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:59.684540 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.703096 23860 rpcz_store.cc:269] 0709 12:12:56.844264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.618988 (+2774724us) service_pool.cc:224] Handling call
0709 12:12:59.649151 (+ 30163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.679767 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 259) took 1299 ms. Trace:
I20260709 12:12:59.646427 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.705871 23779 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.676658 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 243) took 1028 ms. Trace:
I20260709 12:12:59.715083 23854 rpcz_store.cc:276] 0709 12:12:58.648233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.675475 (+1027242us) service_pool.cc:224] Handling call
0709 12:12:59.676598 (+  1123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.703390 24333 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.715824 24333 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:12:59.716293 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 251) took 1369 ms. Trace:
I20260709 12:12:59.716266 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.716806 23781 rpcz_store.cc:276] 0709 12:12:58.344518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.610805 (+1266287us) service_pool.cc:224] Handling call
0709 12:12:59.714217 (+103412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.717303 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 292) took 1371 ms. Trace:
I20260709 12:12:59.717671 23854 rpcz_store.cc:276] 0709 12:12:58.346061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.715900 (+1369839us) service_pool.cc:224] Handling call
0709 12:12:59.717238 (+  1338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.718232 23781 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:12:59.718740 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.718900 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 234) took 2898 ms. Trace:
I20260709 12:12:59.719288 23781 rpcz_store.cc:276] 0709 12:12:56.820415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.717543 (+2897128us) service_pool.cc:224] Handling call
0709 12:12:59.717854 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.718610 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.718842 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.708179 23976 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.665252 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.719934 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.720562 23780 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.721093 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 289) took 1378 ms. Trace:
I20260709 12:12:59.721705 23976 ts_tablet_manager.cc:1434] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.249s	user 0.006s	sys 0.006s
I20260709 12:12:59.722470 23782 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:59.671450 23927 rpcz_store.cc:276] 0709 12:12:58.001116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.669672 (+1668556us) service_pool.cc:224] Handling call
0709 12:12:59.671021 (+  1349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.722807 23976 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:12:59.723001 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 276) took 1195 ms. Trace:
I20260709 12:12:59.723321 23782 rpcz_store.cc:276] 0709 12:12:58.527882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.664988 (+1137106us) service_pool.cc:224] Handling call
0709 12:12:59.722943 (+ 57955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.724216 23926 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:12:59.721465 23859 rpcz_store.cc:276] 0709 12:12:58.342432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.674143 (+1331711us) service_pool.cc:224] Handling call
0709 12:12:59.721032 (+ 46889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.716248 24333 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.725281 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.725905 23859 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.638665 24462 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.727317 23929 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.727264 24498 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:59.673787 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.728107 23932 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.727988 24498 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.728438 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.708698 23785 rpcz_store.cc:276] 0709 12:12:58.380073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.581196 (+1201123us) service_pool.cc:224] Handling call
0709 12:12:59.679696 (+ 98500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.709841 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.729039 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.729530 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 238) took 1082 ms. Trace:
I20260709 12:12:59.729722 23932 raft_consensus.cc:2368] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:12:59.668052 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.711910 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.712250 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.730057 24498 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:12:59.661928 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0080380379997678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 12:12:59.713606 23783 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:59.713959 23856 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.730506 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.668643 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 295) took 1301 ms. Trace:
W20260709 12:12:59.713136 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 286) took 1424 ms (client timeout 1834 ms). Trace:
I20260709 12:12:59.668699 23858 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.700363 24495 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:59.731837 23925 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:12:59.732287 24495 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:12:59.732846 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.733399 23925 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.733207 24495 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.734261 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.734385 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.734861 23925 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.735785 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.731850 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 240) took 1084 ms. Trace:
W20260709 12:12:59.731477 23851 rpcz_store.cc:269] 0709 12:12:58.289456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.646106 (+1356650us) service_pool.cc:224] Handling call
0709 12:12:59.713059 (+ 66953us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.737241 23926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 303) took 1925 ms (client timeout 2090 ms). Trace:
I20260709 12:12:59.736977 23858 rpcz_store.cc:276] 0709 12:12:58.647387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.667816 (+1020429us) service_pool.cc:224] Handling call
0709 12:12:59.731792 (+ 63976us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.736974 23856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 214) took 2276 ms (client timeout 2898 ms). Trace:
I20260709 12:12:59.736476 23925 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.738211 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:12:59.738299 23856 rpcz_store.cc:269] 0709 12:12:57.460204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.675395 (+2215191us) service_pool.cc:224] Handling call
0709 12:12:59.736803 (+ 61408us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.736914 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79089024}
I20260709 12:12:59.730967 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.731065 23932 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.737614 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.736557 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.738898 23858 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.739733 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 242) took 1091 ms. Trace:
I20260709 12:12:59.739653 23929 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.739832 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.731411 23857 rpcz_store.cc:276] 0709 12:12:58.367050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.658129 (+1291079us) service_pool.cc:224] Handling call
0709 12:12:59.668584 (+ 10455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.738818 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.741037 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.741221 23858 rpcz_store.cc:276] 0709 12:12:58.647947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.737972 (+1090025us) service_pool.cc:224] Handling call
0709 12:12:59.739680 (+  1708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.741617 23929 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.729866 23860 rpcz_store.cc:276] 0709 12:12:58.646872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.709605 (+1062733us) service_pool.cc:224] Handling call
0709 12:12:59.729476 (+ 19871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.742197 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.741492 23925 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.740931 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.734943 23930 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.743384 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 235) took 1115 ms. Trace:
I20260709 12:12:59.743804 23851 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.742486 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.708930 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.744922 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 234) took 1117 ms. Trace:
W20260709 12:12:59.668651 23852 rpcz_store.cc:269] 0709 12:12:58.250640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.666997 (+1416357us) service_pool.cc:224] Handling call
0709 12:12:59.668156 (+  1159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.655553 23778 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:12:59.745088 23784 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:59.746364 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 233) took 2926 ms. Trace:
I20260709 12:12:59.742463 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.704939 23899 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.719547 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 236) took 1091 ms. Trace:
I20260709 12:12:59.740983 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.748749 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.758076 23856 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.758455 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 212) took 2298 ms. Trace:
I20260709 12:12:59.748799 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 304) took 1934 ms. Trace:
I20260709 12:12:59.745272 23860 rpcz_store.cc:276] 0709 12:12:58.627977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.742287 (+1114310us) service_pool.cc:224] Handling call
0709 12:12:59.744871 (+  2584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.758765 23856 rpcz_store.cc:276] 0709 12:12:57.459679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.744024 (+2284345us) service_pool.cc:224] Handling call
0709 12:12:59.758398 (+ 14374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.749884 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 237) took 1110 ms. Trace:
I20260709 12:12:59.743780 23857 rpcz_store.cc:276] 0709 12:12:58.628295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.740676 (+1112381us) service_pool.cc:224] Handling call
0709 12:12:59.743324 (+  2648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.750568 23785 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:12:59.704049 23776 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.765751 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 255) took 2604 ms. Trace:
I20260709 12:12:59.765820 23784 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:12:59.766072 23785 rpcz_store.cc:276] 0709 12:12:57.161171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.750028 (+2588857us) service_pool.cc:224] Handling call
0709 12:12:59.750355 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.765475 (+ 15120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.765695 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.751348 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 261) took 1362 ms. Trace:
W20260709 12:12:59.737690 23926 rpcz_store.cc:269] 0709 12:12:57.811532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.510663 (+1699131us) service_pool.cc:224] Handling call
0709 12:12:59.735528 (+224865us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002049d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.737182 (+  1654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488878848}
I20260709 12:12:59.749780 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 291) took 1406 ms. Trace:
I20260709 12:12:59.752097 24462 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.629958 23777 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:12:59.756629 23779 rpcz_store.cc:276] 0709 12:12:56.820161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.745574 (+2925413us) service_pool.cc:224] Handling call
0709 12:12:59.745865 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.746041 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.746081 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:59.746125 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:59.746149 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.746200 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.746303 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:59.759321 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54466.3us
I20260709 12:12:59.759582 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.767088 23931 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:59.712276 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.702944 23928 rpcz_store.cc:276] 0709 12:12:58.187718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.523552 (+1335834us) service_pool.cc:224] Handling call
0709 12:12:59.538117 (+ 14565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.770001 23860 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.702348 23855 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.770782 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.773883 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 293) took 1427 ms. Trace:
I20260709 12:12:59.772593 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.773543 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.775295 23860 rpcz_store.cc:276] 0709 12:12:58.346477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.759336 (+1412859us) service_pool.cc:224] Handling call
0709 12:12:59.773827 (+ 14491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.773979 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.776111 23779 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:12:59.776065 23785 raft_consensus.cc:2368] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:12:59.767217 23853 rpcz_store.cc:276] 0709 12:12:58.342999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.674400 (+1331401us) service_pool.cc:224] Handling call
0709 12:12:59.749708 (+ 75308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.780922 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.782687 23860 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.784425 23975 tablet_bootstrap.cc:654] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:59.781472 23856 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.774503 23899 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:59.768767 24491 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.767331 24498 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:59.787029 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.782140 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.788318 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.788543 24515 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:12:59.759128 23934 rpcz_store.cc:276] 0709 12:12:57.814278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.667547 (+1853269us) service_pool.cc:224] Handling call
0709 12:12:59.748721 (+ 81174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.875294 23784 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:12:59.794561 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.756523 23854 rpcz_store.cc:276] 0709 12:12:58.628525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.718454 (+1089929us) service_pool.cc:224] Handling call
0709 12:12:59.719489 (+  1035us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.765146 23858 rpcz_store.cc:276] 0709 12:12:58.639144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.741955 (+1102811us) service_pool.cc:224] Handling call
0709 12:12:59.749820 (+  7865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.893445 23779 raft_consensus.cc:2368] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:12:59.895841 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.896176 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.773949 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 269) took 1343 ms. Trace:
I20260709 12:12:59.896786 23779 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.805333 24491 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.896783 23784 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:12:59.769364 23783 rpcz_store.cc:276] 0709 12:12:58.388696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.552086 (+1163390us) service_pool.cc:224] Handling call
0709 12:12:59.751190 (+199104us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b5400092254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.751291 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63933696}
I20260709 12:12:59.913712 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.913844 24491 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:12:59.914328 23783 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.776374 23933 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.916625 24507 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.917115 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 244) took 3003 ms. Trace:
I20260709 12:12:59.917399 24507 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:59.917892 24507 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:59.781813 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.919170 24543 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:12:59.920025 23926 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.875988 24498 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:59.921276 24498 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.900234 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:12:59.776425 24462 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:59.903066 23777 rpcz_store.cc:276] 0709 12:12:58.428932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.491910 (+1062978us) service_pool.cc:224] Handling call
0709 12:12:59.771974 (+280064us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:59.902827 23825 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:59.901109 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.922415 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.925182 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.929055 23860 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:12:59.929682 23929 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.925922 23825 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:12:59.781131 23855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 282) took 1559 ms (client timeout 1910 ms). Trace:
I20260709 12:12:59.932381 23825 leader_election.cc:400] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:59.932669 23825 leader_election.cc:403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:59.929114 23779 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:12:59.933676 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.851061 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.934900 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.935935 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:12:59.930579 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 156150us
I20260709 12:12:59.937002 23899 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.925627 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.938745 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.939437 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:12:59.932044 23854 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:12:59.917488 23782 rpcz_store.cc:276] 0709 12:12:56.913653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.724092 (+2810439us) service_pool.cc:224] Handling call
0709 12:12:59.771361 (+ 47269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.776494 (+  5133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.786414 (+  9920us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:59.789075 (+  2661us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:12:59.842213 (+ 53138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.842310 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.916934 (+ 74624us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b540012c5d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:12:59.917045 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102745344}
I20260709 12:12:59.927567 23824 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 310)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 256)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 305)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 258)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 255)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 270)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 311)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 265)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 267)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 273)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 307)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 257)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 253)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 271)
I20260709 12:12:59.942168 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:12:59.951351 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.943342 23899 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.943709 23929 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:12:59.940910 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.921387 24543 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:12:59.951740 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1040 after lost signal to thread 22632
I20260709 12:12:59.924629 24462 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:12:59.988900 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45608us
I20260709 12:12:59.990682 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:12:59.991575 23929 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:12:59.992830 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:12:59.993402 23929 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.993589 23749 leader_election.cc:400] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:12:59.994105 23749 leader_election.cc:403] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:12:59.965620 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 242) took 3100 ms. Trace:
I20260709 12:12:59.990682 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:12:59.960112 23778 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:12:59.984380 23858 raft_consensus.cc:2368] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:12:59.941308 24428 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.998137 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.933113 24515 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.941524 24506 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:59.994617 24462 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:59.948555 24417 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:59.994695 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:12:59.999365 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:12:59.931559 23855 rpcz_store.cc:269] 0709 12:12:58.217074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.651165 (+1434091us) service_pool.cc:224] Handling call
0709 12:12:59.776336 (+125171us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:00.000896 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08000be860 after lost signal to thread 23717
I20260709 12:13:00.047366 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.001854 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:00.048416 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 106308us
I20260709 12:12:59.948242 24489 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.002727 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.003975 23777 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:00.004629 23932 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:00.005110 23928 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.005532 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.005800 23851 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:12:59.997645 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 281) took 1830 ms. Trace:
I20260709 12:13:00.007900 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.009016 23860 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.099555 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.100435 23860 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.078680 23776 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.008460 24543 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.037389 24417 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:12:59.989317 24286 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.050132 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:12:59.997874 24495 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.079061 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.091766 23975 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:13:00.009447 24466 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:00.047526 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.004670 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.086558 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.095010 23824 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 255)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 248)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 249)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 246)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 270)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 311)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 265)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 267)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 273)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 307)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 257)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 253)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 271)
I20260709 12:13:00.009780 24428 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.007506 24493 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.009812 24515 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.101176 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.099647 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 233) took 1478 ms. Trace:
I20260709 12:13:00.037195 24506 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.081439 24489 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.110069 23975 ts_tablet_manager.cc:1403] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.473s	user 0.013s	sys 0.013s
I20260709 12:13:00.093807 24521 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:12:59.945559 24491 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.113862 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:00.115914 23825 leader_election.cc:343] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.117667 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.102993 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:00.130208 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.130815 23781 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.131425 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.132040 23933 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:00.132175 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 270) took 2303 ms. Trace:
I20260709 12:13:00.133033 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.133745 23858 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:00.100812 24507 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.117496 24506 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.106380 24286 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.120159 23934 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:00.120215 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 267) took 2291 ms. Trace:
I20260709 12:13:00.120645 24493 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.106194 24417 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.104422 24543 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.112334 24428 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.111846 24495 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.111081 24498 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.111562 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:12:59.994573 23780 rpcz_store.cc:276] 0709 12:12:56.865242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.721254 (+2856012us) service_pool.cc:224] Handling call
0709 12:12:59.721582 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.721930 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.721962 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.722464 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:12:59.845694 (+123230us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:12:59.895836 (+ 50142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.923962 (+ 28126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:59.924912 (+   950us) raft_consensus.cc:1685] finished
0709 12:12:59.930097 (+  5185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.945772 (+ 15675us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5856,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":68016,"threads_started":1,"wal-append.queue_time_us":177085}
I20260709 12:13:00.148056 24286 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.114632 23931 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:00.169227 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.169492 24498 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.170017 23931 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:00.169679 24417 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:00.170653 24576 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.126104 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 281) took 3194 ms. Trace:
I20260709 12:13:00.171079 24576 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.093461 23853 rpcz_store.cc:276] 0709 12:12:58.164905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.781903 (+1616998us) service_pool.cc:224] Handling call
0709 12:12:59.994958 (+213055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.171878 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 280) took 3252 ms. Trace:
I20260709 12:13:00.168151 23857 raft_consensus.cc:2368] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.129637 24521 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:00.117969 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.172287 23934 rpcz_store.cc:276] 0709 12:12:56.919762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.111367 (+3191605us) service_pool.cc:224] Handling call
0709 12:13:00.119829 (+  8462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.147948 (+ 28119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.171813 (+ 23865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.171777 24493 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.173067 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.173486 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.171312 23932 rpcz_store.cc:276] 0709 12:12:56.932121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.108373 (+3176252us) service_pool.cc:224] Handling call
0709 12:13:00.125604 (+ 17231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.125778 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.125807 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.125852 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.125882 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.125938 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.126050 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.174072 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.122532 23823 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 303)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 276)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 304)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 317)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 318)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 298)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 270)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 311)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 277)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 315)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 265)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 267)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 273)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 307)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 257)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 275)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 253)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 271)
I20260709 12:13:00.174523 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.175072 24521 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:00.175173 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.175766 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.175997 24521 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.176367 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.177222 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.144924 24333 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.178320 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.178553 23854 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.179520 24333 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.179984 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.142171 23929 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:00.180717 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.181123 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.181720 23929 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.170960 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.182003 23783 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:00.184303 23931 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:00.184934 24507 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.185289 23783 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.181725 24495 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.173425 24493 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.182418 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 283) took 3048 ms. Trace:
I20260709 12:13:00.187975 24493 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.157799 23859 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:00.115576 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08001e7c20 after lost signal to thread 23809
I20260709 12:13:00.144595 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 269) took 2315 ms. Trace:
W20260709 12:13:00.188997 23859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 290) took 1846 ms (client timeout 2240 ms). Trace:
I20260709 12:13:00.189191 23785 rpcz_store.cc:276] 0709 12:12:57.828870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.116539 (+2287669us) service_pool.cc:224] Handling call
0709 12:13:00.143738 (+ 27199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.144081 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.144123 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.144189 (+    66us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.144222 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.144309 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.144528 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:00.190016 23859 rpcz_store.cc:269] 0709 12:12:58.342713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.724904 (+1382191us) service_pool.cc:224] Handling call
0709 12:13:00.188934 (+464030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.190078 23975 raft_consensus.cc:359] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.190788 23975 raft_consensus.cc:385] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:00.157199 24543 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:00.191855 23955 debug-util.cc:398] Leaking SignalData structure 0x7b080027fe40 after lost signal to thread 23884
W20260709 12:13:00.192844 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08002d52a0 after lost signal to thread 23958
W20260709 12:13:00.193099 23955 debug-util.cc:398] Leaking SignalData structure 0x7b080023cec0 after lost signal to thread 23959
I20260709 12:13:00.185927 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 272) took 2356 ms. Trace:
I20260709 12:13:00.194151 23776 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:00.194862 23779 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.132537 23930 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:00.132603 23781 rpcz_store.cc:276] 0709 12:12:57.829138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.131362 (+2302224us) service_pool.cc:224] Handling call
0709 12:13:00.131670 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.131844 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.131872 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.131914 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.131949 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.132006 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.132120 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.196398 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 259) took 3028 ms. Trace:
I20260709 12:13:00.116549 23851 rpcz_store.cc:276] 0709 12:12:58.617978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.737382 (+1119404us) service_pool.cc:224] Handling call
0709 12:13:00.094424 (+357042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.197172 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 264) took 1769 ms. Trace:
I20260709 12:13:00.173503 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:00.190867 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 273) took 2361 ms. Trace:
I20260709 12:13:00.150287 23777 rpcz_store.cc:276] 0709 12:12:57.828348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.110839 (+2282491us) service_pool.cc:224] Handling call
0709 12:13:00.119694 (+  8855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.119881 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.119908 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.119954 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.119987 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.120053 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.120158 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.181483 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 239) took 1534 ms. Trace:
I20260709 12:13:00.179363 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.196782 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 287) took 2989 ms. Trace:
I20260709 12:13:00.199968 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 265) took 1772 ms. Trace:
I20260709 12:13:00.196765 23779 rpcz_store.cc:276] 0709 12:12:57.167825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.048868 (+2881043us) service_pool.cc:224] Handling call
0709 12:13:00.194692 (+145824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.195445 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.196209 (+   764us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54000a0850. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:00.196339 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334587776}
I20260709 12:13:00.200953 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.171514 24576 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.199240 23852 rpcz_store.cc:276] 0709 12:12:58.647130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.180466 (+1533336us) service_pool.cc:224] Handling call
0709 12:13:00.181426 (+   960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.174585 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.198262 23785 rpcz_store.cc:276] 0709 12:12:57.829903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.190105 (+2360202us) service_pool.cc:224] Handling call
0709 12:13:00.190426 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.190576 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.190598 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.190634 (+    36us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.190659 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.190708 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.190816 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.203203 23854 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:00.111079 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 279) took 3205 ms. Trace:
I20260709 12:13:00.132998 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.112624 23782 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:00.104916 23926 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:00.203842 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 270) took 1774 ms. Trace:
I20260709 12:13:00.188221 23929 rpcz_store.cc:276] 0709 12:12:57.134208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.181311 (+3047103us) service_pool.cc:224] Handling call
0709 12:13:00.181559 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.182207 (+   648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.182369 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.200560 23925 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:00.200727 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:00.148051 23825 leader_election.cc:343] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.208671 23925 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.203794 23975 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:13:00.176347 23932 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.209440 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 298) took 2399 ms. Trace:
W20260709 12:13:00.209717 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61714.1us
I20260709 12:13:00.197580 23781 rpcz_store.cc:276] 0709 12:12:58.427644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.196394 (+1768750us) service_pool.cc:224] Handling call
0709 12:13:00.196677 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.196844 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.196872 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.196918 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.196950 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.197005 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.197116 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.209883 23925 rpcz_store.cc:276] 0709 12:12:57.810212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.208218 (+2398006us) service_pool.cc:224] Handling call
0709 12:13:00.208491 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.209192 (+   701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.209387 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.209462 23933 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:00.207710 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 262) took 2380 ms. Trace:
I20260709 12:13:00.211656 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 271) took 1782 ms. Trace:
I20260709 12:13:00.211983 23782 rpcz_store.cc:276] 0709 12:12:57.826863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.008561 (+2181698us) service_pool.cc:224] Handling call
0709 12:13:00.088085 (+ 79524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.207374 (+119289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.207647 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.212054 23781 rpcz_store.cc:276] 0709 12:12:58.429414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.210827 (+1781413us) service_pool.cc:224] Handling call
0709 12:13:00.211140 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.211308 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.211337 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.211378 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.211415 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.211470 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.211571 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.213898 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 278) took 1685 ms. Trace:
I20260709 12:13:00.209733 23975 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.214365 23782 rpcz_store.cc:276] 0709 12:12:58.528381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.212995 (+1684614us) service_pool.cc:224] Handling call
0709 12:13:00.213273 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.213437 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.213466 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.213636 (+   170us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.213677 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.213736 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.213840 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.118610 24489 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.118469 24491 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.215778 24491 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.114495 24515 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:00.142900 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 143674us
I20260709 12:13:00.218106 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.174721 23855 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:00.117672 24573 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:00.219278 24573 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.159356 24575 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.219432 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.202339 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 267) took 1773 ms. Trace:
I20260709 12:13:00.220090 24575 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.200408 23777 rpcz_store.cc:276] 0709 12:12:58.427908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.199144 (+1771236us) service_pool.cc:224] Handling call
0709 12:13:00.199439 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.199637 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.199667 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.199714 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.199746 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.199806 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.199915 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.208972 23928 rpcz_store.cc:276] 0709 12:12:56.906157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.004589 (+3098432us) service_pool.cc:224] Handling call
0709 12:13:00.004900 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.093658 (+ 88758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.108729 (+ 15071us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:00.145150 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 154525us
I20260709 12:13:00.222229 24575 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.222263 24466 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.222775 24466 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.223145 24466 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.219689 24573 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:00.208608 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 193) took 2556 ms. Trace:
I20260709 12:13:00.225229 23929 rpcz_store.cc:276] 0709 12:12:57.652367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.207757 (+2555390us) service_pool.cc:224] Handling call
0709 12:13:00.208113 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.208325 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.208352 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.208391 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.208420 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.208468 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.208552 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.226908 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 307) took 2411 ms. Trace:
I20260709 12:13:00.210497 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.228804 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.228925 24495 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.229616 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.188606 23976 tablet_bootstrap.cc:654] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:00.221786 23928 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
W20260709 12:13:00.156137 23751 leader_election.cc:336] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:00.220353 23779 rpcz_store.cc:276] 0709 12:12:58.428444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.201618 (+1773174us) service_pool.cc:224] Handling call
0709 12:13:00.201865 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.202137 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.202288 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.208451 23856 raft_consensus.cc:2376] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:00.177537 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:00.223047 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.235916 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.227203 23929 rpcz_store.cc:276] 0709 12:12:57.815147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.226125 (+2410978us) service_pool.cc:224] Handling call
0709 12:13:00.226432 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.226598 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.226622 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.226662 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.226689 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.226736 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.226854 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.227631 24428 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.231989 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:00.134490 23824 connection.cc:256] RPC call timeout handler was delayed by -1.5397291449999102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 12:13:00.238953 23899 leader_election.cc:400] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.207139 23785 rpcz_store.cc:276] 0709 12:12:58.429169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.203092 (+1773923us) service_pool.cc:224] Handling call
0709 12:13:00.203367 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.203516 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.203541 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.203617 (+    76us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.203643 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.203694 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.203793 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.239301 23899 leader_election.cc:403] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:13:00.239914 23929 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.241341 23899 leader_election.cc:400] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.185573 23823 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56656 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 312)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 323)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 324)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 325)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 326)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 327)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 313)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 328)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 265)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 267)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 329)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 331)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 332)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 333)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 335)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 273)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 334)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 307)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 257)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 263)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 275)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 330)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 253)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 271)
I20260709 12:13:00.216336 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 280) took 1687 ms. Trace:
W20260709 12:13:00.230926 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76868.3us
I20260709 12:13:00.241693 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 311) took 2425 ms. Trace:
I20260709 12:13:00.252207 23929 rpcz_store.cc:276] 0709 12:12:57.816219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.239192 (+2422973us) service_pool.cc:224] Handling call
0709 12:13:00.239755 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.240296 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.241634 (+  1338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.252730 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.254630 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 287) took 1723 ms. Trace:
I20260709 12:13:00.222787 24575 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.221938 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 282) took 1692 ms. Trace:
I20260709 12:13:00.254983 23975 ts_tablet_manager.cc:1434] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.138s	user 0.004s	sys 0.002s
I20260709 12:13:00.241665 23899 leader_election.cc:403] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:13:00.254983 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.260986 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.216357 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 248) took 1876 ms. Trace:
I20260709 12:13:00.255535 23785 rpcz_store.cc:276] 0709 12:12:58.530807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.239515 (+1708708us) service_pool.cc:224] Handling call
0709 12:13:00.254202 (+ 14687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.254366 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.254391 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.254429 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:00.254455 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.254501 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.254583 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.257838 23929 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:13:00.242158 24507 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.268563 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 199) took 2266 ms. Trace:
I20260709 12:13:00.213898 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 279) took 1685 ms. Trace:
I20260709 12:13:00.262203 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.238018 24333 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.242399 24498 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.250305 23927 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:00.268910 23929 rpcz_store.cc:276] 0709 12:12:58.001690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.253410 (+2251720us) service_pool.cc:224] Handling call
0709 12:13:00.253723 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.268302 (+ 14579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.268507 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.224862 24506 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.250813 24417 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.242872 24580 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.199928 23930 rpcz_store.cc:276] 0709 12:12:57.207177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.196034 (+2988857us) service_pool.cc:224] Handling call
0709 12:13:00.196322 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.196478 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.196503 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.196543 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.196574 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.196626 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.196725 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.163468 24462 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:00.231809 23783 rpcz_store.cc:276] 0709 12:12:57.829641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.184805 (+2355164us) service_pool.cc:224] Handling call
0709 12:13:00.185111 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.185688 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.185871 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.216951 24493 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.328433 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:00.328786 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 87655.9us
I20260709 12:13:00.329118 24493 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.329777 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:00.330051 23824 connection.cc:256] RPC call timeout handler was delayed by -1.4893291449998287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 12:13:00.330487 23824 connection.cc:256] RPC call timeout handler was delayed by -1.4551291450000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 12:13:00.303763 24581 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.326520 23900 leader_election.cc:400] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.330952 24581 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.331416 24581 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.332094 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:00.333204 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:00.332074 23899 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.335291 23900 leader_election.cc:400] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.302883 24578 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.336194 24578 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.336474 23900 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:00.241803 24574 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:13:00.338615 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.317988 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.339244 24574 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.339921 24574 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.340047 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.265143 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 302) took 2451 ms. Trace:
I20260709 12:13:00.340771 23928 rpcz_store.cc:276] 0709 12:12:57.811255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.221274 (+2410019us) service_pool.cc:224] Handling call
0709 12:13:00.221598 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.237644 (+ 16046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.263050 (+ 25406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.250849 24573 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.344090 23860 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.344282 23899 leader_election.cc:400] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.344578 23899 leader_election.cc:403] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:00.345856 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.346055 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 241) took 1698 ms. Trace:
I20260709 12:13:00.346813 23860 rpcz_store.cc:276] 0709 12:12:58.647639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.261941 (+1614302us) service_pool.cc:224] Handling call
0709 12:13:00.345852 (+ 83911us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b54002f4ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:00.345993 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114469760}
I20260709 12:13:00.343004 23900 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:00.348021 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.348309 24493 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.330556 23857 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.336551 24578 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.256850 24521 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.256265 24575 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.340070 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.326586 23975 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
I20260709 12:13:00.324579 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.343302 24573 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:00.331442 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3822291449998829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 12:13:00.348670 23860 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:00.349804 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.348693 24493 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.351881 23853 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.307238 24333 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:00.293812 24507 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.293965 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.352340 23785 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.324576 24498 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.306958 24580 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.306763 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:00.362152 23824 connection.cc:256] RPC call timeout handler was delayed by -1.2733291449999342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
W20260709 12:13:00.371030 23824 connection.cc:256] RPC call timeout handler was delayed by -1.2355291449998731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 12:13:00.325745 24417 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:00.371512 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1100911599999108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
W20260709 12:13:00.371978 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 117071us
I20260709 12:13:00.372608 24573 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.372833 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.373054 24573 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.373036 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.374382 24577 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.374784 24577 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.375145 24577 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.377990 23751 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.379539 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 289) took 1848 ms. Trace:
I20260709 12:13:00.365224 23926 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.366619 24493 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.363214 24575 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.364233 24521 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.364137 24578 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.380540 23785 rpcz_store.cc:276] 0709 12:12:58.531283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.266438 (+1735155us) service_pool.cc:224] Handling call
0709 12:13:00.266790 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.367190 (+100400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.379330 (+ 12140us) spinlock_profiling.cc:243] Waited 85.3 ms on lock 0x7b5400425ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.379481 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196253952}
I20260709 12:13:00.383912 23824 leader_election.cc:400] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:00.382136 24428 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.266288 23776 rpcz_store.cc:276] 0709 12:12:58.340324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.194943 (+1854619us) service_pool.cc:224] Handling call
0709 12:13:00.195271 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.195442 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.195472 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.195520 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.195560 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.216153 (+ 20593us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.216288 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.255775 23782 rpcz_store.cc:276] 0709 12:12:58.528852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.215495 (+1686643us) service_pool.cc:224] Handling call
0709 12:13:00.215832 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.215995 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.216024 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.216067 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.216103 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.216164 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.216275 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.378509 24286 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.363865 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.376782 24417 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.385231 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.359468 23932 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:00.368158 24588 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.369050 23852 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.368819 24507 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.412103 23860 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:00.387166 23824 leader_election.cc:403] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:00.420657 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.376492 23934 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:00.394017 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49861.7us
I20260709 12:13:00.382617 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 316) took 2010 ms. Trace:
I20260709 12:13:00.370687 24580 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.262733 23777 rpcz_store.cc:276] 0709 12:12:58.529322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.221087 (+1691765us) service_pool.cc:224] Handling call
0709 12:13:00.221436 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.221600 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.221626 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.221668 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.221699 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.221763 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.221881 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.389976 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.293759 23781 rpcz_store.cc:276] 0709 12:12:58.528614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.213201 (+1684587us) service_pool.cc:224] Handling call
0709 12:13:00.213444 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.213586 (+   142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.213610 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.213648 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.213683 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.213735 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.213840 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.429430 23751 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.415527 23784 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.421833 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.386292 24521 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.431459 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.432183 23860 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:00.395346 23783 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.434780 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 266) took 2606 ms. Trace:
I20260709 12:13:00.416671 24573 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.414263 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.418313 24588 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.468883 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 201) took 2466 ms. Trace:
I20260709 12:13:00.387079 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.423657 24585 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.427855 24428 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:00.428287 24417 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.415354 24489 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.489810 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.490897 24489 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.443351 23784 rpcz_store.cc:276] 0709 12:12:57.828066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.085552 (+2257486us) service_pool.cc:224] Handling call
0709 12:13:00.085898 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.430297 (+344399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.434567 (+  4270us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54001ee454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.434714 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757610880}
I20260709 12:13:00.491097 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.445268 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:00.491607 24576 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.492360 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.493490 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.493768 24576 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.493953 24594 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.494382 24594 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.494545 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.424845 23930 rpcz_store.cc:276] 0709 12:12:58.372115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.370315 (+1998200us) service_pool.cc:224] Handling call
0709 12:13:00.381202 (+ 10887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.382383 (+  1181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.382559 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.469581 23929 rpcz_store.cc:276] 0709 12:12:58.002248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.296501 (+2294253us) service_pool.cc:224] Handling call
0709 12:13:00.296797 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.296995 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:13:00.371854 (+ 74859us) log.cc:844] Serialized 14 byte log entry
0709 12:13:00.468529 (+ 96675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.468595 (+    66us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.468634 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.468699 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.468820 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":518,"raft.queue_time_us":3590,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":132,"thread_start_us":170342,"threads_started":2}
W20260709 12:13:00.484607 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 63754.1us
I20260709 12:13:00.440213 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.435544 24590 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:00.471151 23859 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:00.503768 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.472332 24286 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.441071 24462 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:00.488742 24491 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.509330 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 292) took 1664 ms. Trace:
W20260709 12:13:00.493783 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48590.7us
I20260709 12:13:00.495302 23931 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.496454 23852 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
W20260709 12:13:00.476943 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 256217us
I20260709 12:13:00.498472 24594 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.416950 24498 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.499881 24589 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.502139 24581 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.473413 24588 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.500296 24598 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.470760 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 265) took 2643 ms. Trace:
I20260709 12:13:00.439152 24495 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.500628 24417 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.473028 24592 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.416759 24543 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.487368 24466 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.437335 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 299) took 2626 ms. Trace:
I20260709 12:13:00.387365 24333 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.505707 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 295) took 2898 ms. Trace:
I20260709 12:13:00.506256 24506 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.506013 24493 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.506794 24576 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.509895 23824 leader_election.cc:400] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:00.510944 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.558864 23934 rpcz_store.cc:276] 0709 12:12:57.607526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.198608 (+2591082us) service_pool.cc:224] Handling call
0709 12:13:00.198915 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.429691 (+230776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.505436 (+ 75745us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000f69d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.505635 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381684096}
I20260709 12:13:00.472235 23928 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.559785 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.438557 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 301) took 2627 ms. Trace:
I20260709 12:13:00.516737 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.561218 24417 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.562247 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.562999 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.563410 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.563611 23852 raft_consensus.cc:2376] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:00.564517 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.564618 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.565171 23852 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:13:00.561229 23925 rpcz_store.cc:276] 0709 12:12:57.811013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.211896 (+2400883us) service_pool.cc:224] Handling call
0709 12:13:00.212662 (+   766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.213949 (+  1287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.213978 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.214461 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.367519 (+153058us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:13:00.371668 (+  4149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.379275 (+  7607us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.438353 (+ 59078us) raft_consensus.cc:1685] finished
0709 12:13:00.438369 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.438494 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":594,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"thread_start_us":146420,"threads_started":1,"wal-append.queue_time_us":178569}
I20260709 12:13:00.566745 24580 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.567632 24576 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.567962 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 285) took 3396 ms. Trace:
I20260709 12:13:00.568187 24576 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.568338 23931 rpcz_store.cc:276] 0709 12:12:57.171860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.184934 (+3013074us) service_pool.cc:224] Handling call
0709 12:13:00.185212 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.536607 (+351395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.567726 (+ 31119us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400203ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.567901 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712639744}
I20260709 12:13:00.518273 23857 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.572507 24577 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.573261 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.573973 23857 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.574102 24507 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.551887 24598 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:00.576359 24493 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.576928 24466 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.576828 24598 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.518647 23853 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:00.578050 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.519721 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 285) took 1989 ms. Trace:
W20260709 12:13:00.519938 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 290380us
I20260709 12:13:00.489024 23927 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:13:00.579252 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.578658 23779 rpcz_store.cc:276] 0709 12:12:58.530136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.230864 (+1700728us) service_pool.cc:224] Handling call
0709 12:13:00.231172 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.231463 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.231490 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.231928 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.371084 (+139156us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.371118 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.371177 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.519469 (+148292us) raft_consensus.cc:1685] finished
0709 12:13:00.519488 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.519660 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62972,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6883,"thread_start_us":110589,"threads_started":1,"wal-append.queue_time_us":251271}
I20260709 12:13:00.524755 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.527354 23975 tablet_bootstrap.cc:654] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:00.517189 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.523149 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 324) took 1589 ms. Trace:
I20260709 12:13:00.536392 23933 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.580503 23851 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.538127 23749 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.524583 24600 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.527305 24515 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.529558 24491 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:00.550082 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 120725us
W20260709 12:13:00.591732 23751 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:56648 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:00.560829 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.565989 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.556972 24575 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.524864 23783 rpcz_store.cc:276] 0709 12:12:58.843133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.359132 (+1515999us) service_pool.cc:224] Handling call
0709 12:13:00.381203 (+ 22071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.469528 (+ 88325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.507914 (+ 38386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.580470 23856 raft_consensus.cc:2410] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:13:00.580976 23824 leader_election.cc:403] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:00.581411 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.579214 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.567643 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.581121 23932 rpcz_store.cc:276] 0709 12:12:58.933949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.443298 (+1509349us) service_pool.cc:224] Handling call
0709 12:13:00.507890 (+ 64592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.522757 (+ 14867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.522798 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.522849 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.522892 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.522956 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.523087 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.552386 23926 rpcz_store.cc:276] 0709 12:12:57.810486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.208256 (+2397770us) service_pool.cc:224] Handling call
0709 12:13:00.208624 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.387650 (+179026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.437104 (+ 49454us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002b7354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.437274 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329086976}
I20260709 12:13:00.585819 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 300) took 2775 ms. Trace:
I20260709 12:13:00.593631 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.594744 23860 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.599257 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 262) took 3680 ms. Trace:
I20260709 12:13:00.574754 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.596504 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.578434 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:00.591329 24466 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.596774 23933 rpcz_store.cc:276] 0709 12:12:57.810747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.211218 (+2400471us) service_pool.cc:224] Handling call
0709 12:13:00.211490 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.580804 (+369314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.585529 (+  4725us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54002431d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.585748 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731633536}
I20260709 12:13:00.602175 24578 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.603236 23855 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.596161 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:00.596774 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 87053.5us
I20260709 12:13:00.574705 24577 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.599572 23860 rpcz_store.cc:276] 0709 12:12:56.919161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.594306 (+3675145us) service_pool.cc:224] Handling call
0709 12:13:00.594597 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.599033 (+  4436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.599207 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.605790 23825 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:00.597848 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.596057 24506 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.607159 24586 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.549376 23778 rpcz_store.cc:276] 0709 12:12:57.827612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.047108 (+2219496us) service_pool.cc:224] Handling call
0709 12:13:00.047462 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.147088 (+ 99626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.147119 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.151101 (+  3982us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.370181 (+219080us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.370233 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.381531 (+ 11298us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.441306 (+ 59775us) raft_consensus.cc:1685] finished
0709 12:13:00.450283 (+  8977us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.470699 (+ 20416us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136054,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":261318,"threads_started":2,"wal-append.queue_time_us":223110}
I20260709 12:13:00.608943 24543 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.606241 24466 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.596093 24597 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:00.684689 24466 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.596482 24428 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:13:00.595921 24575 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.619097 23751 leader_election.cc:400] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:00.620983 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:00.688845 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69933.8us
W20260709 12:13:00.689002 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68090.9us
W20260709 12:13:00.689606 23749 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.690264 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.690820 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.591687 24521 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:00.656677 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58868.3us
I20260709 12:13:00.696095 23900 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:00.611403 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.610948 23859 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:00.598003 24498 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:00.681876 24286 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.701444 24580 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.701786 24286 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.702199 24286 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.702436 24498 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.593959 24600 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.682163 24491 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.613405 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 260) took 3696 ms. Trace:
I20260709 12:13:00.611965 23852 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:00.694835 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.615202 23856 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.718709 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 263) took 3796 ms. Trace:
I20260709 12:13:00.696764 24577 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:00.614600 24507 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:00.719527 24577 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.719971 24577 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.700161 24592 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.700304 24543 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.721736 24543 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:00.722157 24543 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.700591 24417 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.696697 24589 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:00.701193 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 91) took 2082 ms. Trace:
I20260709 12:13:00.614928 23855 raft_consensus.cc:1240] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.728502 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 273) took 3512 ms. Trace:
I20260709 12:13:00.728935 23855 rpcz_store.cc:276] 0709 12:12:57.216388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.614435 (+3398047us) service_pool.cc:224] Handling call
0709 12:13:00.614747 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.728159 (+113412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.728428 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.730198 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.730854 23855 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.611896 23858 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:00.704375 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 93032.4us
I20260709 12:13:00.679738 24506 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:00.732134 24506 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.680902 24586 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.733124 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:00.733486 24586 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:00.708068 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 102354us
I20260709 12:13:00.681173 24601 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.615612 24515 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.724918 23681 rpcz_store.cc:276] 0709 12:12:58.619724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.398422 (+778698us) service_pool.cc:224] Handling call
0709 12:13:00.701118 (+1302696us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:59.805214 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:59.842554 (+ 37340us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:59.842585 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:12:59.842605 (+    20us) tablet.cc:662] Decoding operations
0709 12:12:59.843041 (+   436us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:59.843145 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:59.843176 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:12:59.843533 (+   357us) tablet.cc:701] Row locks acquired
0709 12:12:59.843553 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:12:59.843750 (+   197us) write_op.cc:270] Start()
0709 12:12:59.894750 (+ 51000us) write_op.cc:276] Timestamp: P: 1783599179843694 usec, L: 0
0709 12:12:59.895635 (+   885us) op_driver.cc:348] REPLICATION: starting
0709 12:12:59.911816 (+ 16181us) log.cc:844] Serialized 922 byte log entry
0709 12:13:00.267873 (+356057us) op_driver.cc:464] REPLICATION: finished
0709 12:13:00.501030 (+233157us) write_op.cc:301] APPLY: starting
0709 12:13:00.501108 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:00.501233 (+   125us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:00.501385 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:00.538362 (+ 36977us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:00.556833 (+ 18471us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:13:00.583865 (+ 27032us) write_op.cc:312] APPLY: finished
0709 12:13:00.606366 (+ 22501us) log.cc:844] Serialized 28 byte log entry
0709 12:13:00.680970 (+ 74604us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.681958 (+   988us) write_op.cc:454] Released schema lock
0709 12:13:00.681999 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:00.702178 (+ 20179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.702216 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20937,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":232858,"apply.run_cpu_time_us":2782,"apply.run_wall_time_us":194893,"mutex_wait_us":3038,"num_ops":2,"prepare.queue_time_us":139373,"prepare.run_cpu_time_us":39585,"prepare.run_wall_time_us":288602,"raft.queue_time_us":155001,"raft.run_cpu_time_us":23996,"raft.run_wall_time_us":168450,"replication_time_us":372075,"thread_start_us":505241,"threads_started":4,"wal-append.queue_time_us":180329}]]}
I20260709 12:13:00.719059 23856 rpcz_store.cc:276] 0709 12:12:56.922149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.595178 (+3673029us) service_pool.cc:224] Handling call
0709 12:13:00.615047 (+ 19869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.717615 (+102568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.718471 (+   856us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54002f53d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:00.718645 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44970880}
I20260709 12:13:00.737929 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 228) took 2516 ms. Trace:
I20260709 12:13:00.737742 24515 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.738296 23856 rpcz_store.cc:276] 0709 12:12:58.221384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.737034 (+2515650us) service_pool.cc:224] Handling call
0709 12:13:00.737352 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.737515 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.737541 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.737579 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.737606 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.737756 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.737873 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.734788 23858 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:13:00.740621 23824 leader_election.cc:343] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.741132 23824 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.740571 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 279) took 2576 ms. Trace:
I20260709 12:13:00.742460 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:00.751564 24506 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.743808 24515 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:13:00.752439 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 57666.7us
I20260709 12:13:00.754153 24515 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:00.754956 24515 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.755784 24601 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.751335 23858 rpcz_store.cc:276] 0709 12:12:58.164385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.734232 (+2569847us) service_pool.cc:224] Handling call
0709 12:13:00.734561 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.740297 (+  5736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.740514 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.744921 24507 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.750005 24586 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.756340 24601 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:00.759958 24507 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:00.760425 24507 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.760095 24601 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.761309 24507 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.754171 24506 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.755348 24515 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.744240 24592 consensus_queue.cc:1059] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.752264 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.762888 24507 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.764052 24462 raft_consensus.cc:3055] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:00.764416 24462 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:00.765062 23824 leader_election.cc:343] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.765604 23824 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.765271 24462 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.763442 24515 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.767553 24592 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.768688 24590 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:00.766551 24462 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:00.769129 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.770185 24586 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.770524 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:00.771124 24586 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.769013 24590 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.772882 24590 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:00.773778 23824 leader_election.cc:343] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.774353 23824 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:00.776999 24590 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:00.778143 24586 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:00.778913 24586 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.779246 24586 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.779618 24586 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.781709 24515 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.753099 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:00.785202 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 301) took 1132 ms. Trace:
I20260709 12:13:00.785729 23858 rpcz_store.cc:276] 0709 12:12:59.653205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.763632 (+1110427us) service_pool.cc:224] Handling call
0709 12:13:00.784614 (+ 20982us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.784817 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.784845 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.784899 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.784935 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.785000 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.785131 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.787957 24601 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.788570 24601 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:00.788800 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.790737 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:00.791388 23860 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:13:00.792027 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 213) took 3332 ms. Trace:
I20260709 12:13:00.792356 23860 rpcz_store.cc:276] 0709 12:12:57.459932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.692008 (+3232076us) service_pool.cc:224] Handling call
0709 12:13:00.791189 (+ 99181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.791759 (+   570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.791970 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.792513 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:00.788293 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:00.789731 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:00.800743 23824 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:00.825841 24515 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.826499 24515 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:13:00.826843 24515 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.827153 24515 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:00.787451 23858 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:00.829921 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 306) took 1174 ms. Trace:
I20260709 12:13:00.830292 23858 rpcz_store.cc:276] 0709 12:12:59.655817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.786948 (+1131131us) service_pool.cc:224] Handling call
0709 12:13:00.787270 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.829640 (+ 42370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.829860 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.819257 24333 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:00.832878 24590 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.703488 24498 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.833408 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 271) took 3004 ms. Trace:
I20260709 12:13:00.833841 24495 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:00.834295 24495 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.833806 23780 rpcz_store.cc:276] 0709 12:12:57.829395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.172206 (+2342811us) service_pool.cc:224] Handling call
0709 12:13:00.172586 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.306610 (+134024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.325830 (+ 19220us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.386726 (+ 60896us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.600927 (+214201us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.604858 (+  3931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.611354 (+  6496us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.833182 (+221828us) raft_consensus.cc:1685] finished
0709 12:13:00.833211 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.833344 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11151,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":65,"thread_start_us":278990,"threads_started":2,"wal-append.queue_time_us":290379}
I20260709 12:13:00.835443 24495 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.836376 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 253) took 1082 ms. Trace:
I20260709 12:13:00.836571 24495 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.836730 23860 rpcz_store.cc:276] 0709 12:12:59.753677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.793332 (+1039655us) service_pool.cc:224] Handling call
0709 12:13:00.835921 (+ 42589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.836088 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.836112 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.836150 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.836178 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.836226 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.836322 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.838198 23860 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:00.840634 23859 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.842033 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 276) took 2696 ms. Trace:
I20260709 12:13:00.842427 23859 rpcz_store.cc:276] 0709 12:12:58.146007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.701076 (+2555069us) service_pool.cc:224] Handling call
0709 12:13:00.793128 (+ 92052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.841137 (+ 48009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.841801 (+   664us) spinlock_profiling.cc:243] Waited 88.8 ms on lock 0x7b540037ee50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:00.841963 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204239232}
I20260709 12:13:00.843799 24575 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:00.844372 24581 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.839398 23860 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:00.821713 23824 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:00.847240 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46530.1us
I20260709 12:13:00.716939 24580 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:00.793488 24589 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.716789 24491 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.845402 24506 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:00.860498 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.860669 23857 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:13:00.844753 23851 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:00.716915 24493 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.140s
I20260709 12:13:00.863303 23855 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:00.864125 24607 raft_consensus.cc:2764] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:00.849586 24333 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:00.866671 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.866297 23976 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:13:00.869764 23976 ts_tablet_manager.cc:1403] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 1.147s	user 0.029s	sys 0.022s
I20260709 12:13:00.874445 24466 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.866695 24462 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:13:00.857903 24601 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.858274 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 264) took 3930 ms. Trace:
I20260709 12:13:00.881215 23900 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:00.856472 24580 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:00.843799 24590 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:13:00.881438 23860 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:00.884337 23860 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:00.859790 24589 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:00.855630 24592 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:13:00.867565 24506 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.867106 24521 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.880251 23859 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:00.889997 24597 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:00.886471 24580 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.712324 23854 rpcz_store.cc:276] 0709 12:12:56.917270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.580275 (+3663005us) service_pool.cc:224] Handling call
0709 12:13:00.580547 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.591531 (+ 10984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.593511 (+  1980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.600044 (+  6533us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.600652 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.610725 (+ 10073us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.613308 (+  2583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.972908 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 229) took 2684 ms. Trace:
I20260709 12:13:00.992057 23856 rpcz_store.cc:276] 0709 12:12:58.288346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.739189 (+2450843us) service_pool.cc:224] Handling call
0709 12:13:00.739486 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.856226 (+116740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.856256 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.856295 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.856324 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.957940 (+101616us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54002f6cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:00.972704 (+ 14764us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.972837 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268071680}
I20260709 12:13:00.958709 23860 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.993420 24489 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:00.887936 24462 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.993932 23860 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:00.925472 24428 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:00.866048 24599 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:00.912351 24585 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:00.997893 24286 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.000602 24286 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.926637 24600 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:00.858814 24594 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.981805 23976 raft_consensus.cc:359] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.914417 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 277) took 2766 ms. Trace:
I20260709 12:13:00.880584 24573 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 12:13:00.947121 24506 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:00.924932 24543 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.879910 23853 rpcz_store.cc:276] 0709 12:12:56.928333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.618025 (+3689692us) service_pool.cc:224] Handling call
0709 12:13:00.696127 (+ 78102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.790585 (+ 94458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.793895 (+  3310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:00.795541 (+  1646us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:00.845735 (+ 50194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.847745 (+  2010us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.857044 (+  9299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:00.974241 24466 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.001998 23976 raft_consensus.cc:385] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:01.008165 23976 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:13:01.009086 24543 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.008904 23976 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.004580 23860 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:01.003950 23857 rpcz_store.cc:276] 0709 12:12:58.146262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.716280 (+2570018us) service_pool.cc:224] Handling call
0709 12:13:00.847848 (+131568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.868803 (+ 20955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.912817 (+ 44014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:01.003890 24585 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.005713 24462 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:00.991920 24589 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.883083 24576 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.016232 23852 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.016185 23857 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.028677 24493 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.029274 24581 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.011574 24585 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.994488 24428 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:00.993798 24489 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.015921 24462 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.021248 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.021596 24515 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.023228 24588 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.024986 24594 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:01.026141 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.014251 24593 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:13:01.020066 24578 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:01.014940 24495 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:13:01.035884 24586 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.977372 24491 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:00.985181 24577 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.077162 24581 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:01.053398 24580 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.078802 23976 ts_tablet_manager.cc:1434] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.209s	user 0.009s	sys 0.017s
W20260709 12:13:01.081389 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 218442us
I20260709 12:13:01.005367 24498 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.076s
I20260709 12:13:01.063323 24493 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:01.055850 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 161313us
W20260709 12:13:01.091920 23751 leader_election.cc:336] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.558s (SENT)
W20260709 12:13:01.092389 23751 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.604s (SENT)
W20260709 12:13:01.093567 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 271605us
I20260709 12:13:01.094010 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:01.098796 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.099895 24586 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.100502 23858 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.100853 24577 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.100927 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.101429 24489 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.102617 24489 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.103189 24580 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.098575 24495 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.096064 24607 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.104404 24495 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.094619 24594 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.105574 24594 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.076644 24597 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.105263 24495 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.096483 23976 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:13:01.090040 24428 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.109393 24581 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.095890 24543 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:13:01.113487 24495 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.115523 23779 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.115705 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.116838 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.117148 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.117753 23784 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:01.118260 23779 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.118288 23975 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:13:01.103215 24515 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.119884 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.119946 23975 ts_tablet_manager.cc:1403] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 0.794s	user 0.040s	sys 0.037s
I20260709 12:13:01.122037 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:01.096192 24599 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:13:01.133090 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.125690 23899 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:01.131482 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.134265 23929 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:01.135350 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.135569 23899 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:01.136165 23929 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.131994 23779 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.148332 24543 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:01.149010 24521 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.149274 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:01.150269 24590 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.150350 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:01.150985 24578 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:01.151180 24592 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:01.151501 24578 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.151819 23784 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:01.152280 24592 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.152567 24578 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.153180 24592 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.153208 24621 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:01.154659 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.155874 24630 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:01.155817 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.157232 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.157831 23929 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.105988 24506 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.161068 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.162185 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.163502 24627 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:01.118883 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.123942 24428 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.156379 24507 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:01.123330 24588 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:01.111182 24586 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:13:01.147102 24581 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.169334 24599 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:13:01.164379 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.171063 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.173092 24627 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.132696 24498 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.178076 23976 tablet_bootstrap.cc:654] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:01.169224 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.178643 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.143410 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.181257 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.147933 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.189416 23779 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.189699 23927 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.173712 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.190621 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.191447 23927 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.190583 23928 raft_consensus.cc:2368] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:01.192365 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.192674 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.193461 23928 raft_consensus.cc:2368] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:01.194310 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.196157 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.197329 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.197947 23784 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:01.183065 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.183905 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.209174 23784 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:13:01.215402 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.181905 24507 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.216683 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.217515 23784 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.196983 23778 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.191574 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.214942 23925 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.160415 24594 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:01.220083 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49134.7us
I20260709 12:13:01.205008 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.222038 23785 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:01.164029 24612 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:13:01.167765 24629 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.050s
W20260709 12:13:01.223901 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 56892.1us
I20260709 12:13:01.188455 24588 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:13:01.225579 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.226404 23779 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
W20260709 12:13:01.226758 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.205026 23926 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:01.215541 24576 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.244010 23824 leader_election.cc:400] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:01.249598 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:01.247112 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.223409 23927 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.248934 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.215498 24599 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:13:01.256332 23751 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.257452 23751 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.258039 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:01.260371 24619 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:13:01.259886 24507 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.261572 24613 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:01.262120 24613 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:01.262903 24613 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.264164 24589 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:01.264343 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.253842 24618 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.265054 23851 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 0.
I20260709 12:13:01.265192 24618 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.267343 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.268975 23751 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:13:01.269299 23751 leader_election.cc:403] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:13:01.266270 24506 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.272790 24622 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3.
I20260709 12:13:01.273072 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.273677 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.274184 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.274824 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.275193 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.271278 23928 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.276480 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.277036 23780 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.277554 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.278257 23975 raft_consensus.cc:359] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.278653 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.278914 23975 raft_consensus.cc:385] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:01.279194 23975 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:13:01.249590 24600 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:13:01.315883 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.319269 23860 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.295310 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.181831 24631 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 12:13:01.294744 23975 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:01.326370 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 169966us
I20260709 12:13:01.258294 24576 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:01.329937 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.330148 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.331374 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.332504 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.333534 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.334525 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.335779 24600 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.255141 24627 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:01.324415 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52522.8us
W20260709 12:13:01.338290 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.280725 24589 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:01.339915 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.345230 24589 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:01.345772 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.357640 23927 raft_consensus.cc:2410] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:13:01.359138 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.345270 23975 ts_tablet_manager.cc:1434] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.222s	user 0.003s	sys 0.009s
I20260709 12:13:01.344421 23751 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.363022 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.363407 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.363719 24618 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:01.364403 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.364668 23929 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:13:01.364857 24622 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.338690 24621 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:13:01.363019 24466 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:01.366454 24466 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:01.364624 23751 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.366883 24466 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.367348 24622 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.344233 24632 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:13:01.368757 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.368944 23751 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.369412 23852 raft_consensus.cc:2376] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3. Re-sending same reply.
I20260709 12:13:01.369621 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.370224 23932 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:01.370311 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.370121 24622 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.371140 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.372105 23932 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.361924 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.373134 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.373395 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.374626 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.375443 23932 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.358871 24589 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:01.364305 24623 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:13:01.372234 23751 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:01.435281 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46633.1us
I20260709 12:13:01.437357 23928 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:01.438496 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.439096 23928 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.360352 24627 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.440016 23975 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Bootstrap starting.
W20260709 12:13:01.441008 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68820.7us
I20260709 12:13:01.431660 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.431833 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.448513 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:01.450107 23751 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.447505 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.360212 23925 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:01.463092 23932 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.464135 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.464939 23932 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.465790 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.473510 24428 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:01.377179 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.434258 24618 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.434053 24630 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.505119 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:01.461894 24575 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:01.492192 24576 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:01.508275 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.508994 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.474552 23931 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:01.511458 24428 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.505879 24622 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:01.519640 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.522070 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.512212 24630 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:01.529850 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 235243us
I20260709 12:13:01.516207 24632 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:01.523353 23852 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.535148 23856 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:01.513551 24618 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.540609 23856 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:01.544130 24580 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.550776 24632 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.536828 24507 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 12:13:01.606340 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.558962 23927 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:13:01.605722 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68678.5us
I20260709 12:13:01.573612 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.584594 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.609877 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.549750 24466 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.584558 24580 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.620394 23932 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:01.637820 23782 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:01.622612 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:01.589799 24630 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.645800 24466 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.643908 24648 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.646823 24648 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.648072 23931 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.648505 23975 tablet_bootstrap.cc:654] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:01.648934 24648 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.643875 23925 raft_consensus.cc:2410] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:01.650523 23931 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.651484 23925 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:01.652475 24632 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.642648 24618 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:01.608381 24428 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.616117 24640 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:13:01.650009 23932 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.671969 23932 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.660899 23776 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 0.
I20260709 12:13:01.656260 24580 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.739841 24580 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.748586 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.749583 23859 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.750128 23927 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:13:01.751338 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.754145 24493 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.770187 24600 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:01.752519 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 271) took 1829 ms. Trace:
I20260709 12:13:01.778702 24594 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.775168 23858 rpcz_store.cc:276] 0709 12:12:59.921513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.832213 (+910700us) service_pool.cc:224] Handling call
0709 12:13:00.832602 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.100876 (+268274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.731480 (+630604us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54004141d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:01.751203 (+ 19723us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615682432}
I20260709 12:13:01.780233 24521 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.780812 24521 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.781625 24521 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.779433 24493 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.783675 24521 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.784274 24628 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.784654 24628 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.785519 24573 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:01.786056 23928 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:01.786433 24597 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.786860 23925 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:01.788280 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.786784 24597 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.788923 23933 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:01.789356 24597 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.785907 24573 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.790781 24597 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.790987 24573 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:13:01.794915 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.796218 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:01.797060 23930 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.801033 24589 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.803804 24589 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.806610 24495 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.807184 24495 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:01.808074 24495 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.809322 24495 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.812224 24599 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:01.819628 24630 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.827239 23860 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:01.832362 24599 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:01.836792 24599 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.838011 24599 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:01.856797 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.858700 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:01.859754 24573 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.898447 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.899194 23784 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.899783 23785 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:01.900257 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.901000 23784 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.898453 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.902098 23779 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.903090 23785 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:01.911880 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.912900 23780 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.914806 24581 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.918745 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:01.919660 23856 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:01.915364 24581 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:01.924232 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 313007us
I20260709 12:13:01.927050 23779 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.933887 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.934607 23779 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:01.907717 24594 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.953076 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.931926 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.955297 23782 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.942313 24594 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.956802 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.957579 23784 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.926740 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:01.958204 23779 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:01.958185 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:01.961606 23855 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:01.961850 23976 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:13:01.962392 23976 ts_tablet_manager.cc:1403] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.866s	user 0.050s	sys 0.060s
I20260709 12:13:01.941622 24585 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:01.942688 24495 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:01.966179 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.967180 23779 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:01.978586 23853 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.983732 24581 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:01.989362 24641 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:01.956234 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:01.995016 24607 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:01.957901 24489 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.996254 24628 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.999445 23927 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:01.999673 24628 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:01.960546 23852 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.002446 23824 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:02.004560 23929 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:13:02.005774 23929 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:01.984368 24639 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.000412 24607 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.017930 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:01.991173 24585 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:01.989411 24495 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.020496 23976 raft_consensus.cc:359] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.021023 24585 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.022336 24594 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.023066 23926 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.023216 23853 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:13:02.023650 24639 consensus_queue.cc:1243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:02.024171 24639 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:02.024471 24639 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.021212 23976 raft_consensus.cc:385] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:02.027383 23976 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
W20260709 12:13:02.029871 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.023669 23824 leader_election.cc:343] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:02.027153 23777 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:02.034633 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:02.035498 23779 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:02.037225 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:02.038165 23934 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 2: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:02.041302 23856 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.042444 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.045424 23778 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:13:02.051194 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.002532 24489 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.027602 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.047391 23780 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:02.061707 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.062366 23779 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:02.064222 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.054955 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.064958 23856 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:02.053424 24652 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:02.023327 24586 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:02.059396 23976 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.071925 23782 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:02.073473 24654 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:02.073920 24654 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.074715 24654 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:02.080139 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.502s (SENT)
I20260709 12:13:02.080586 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:02.084534 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.085202 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.027s
W20260709 12:13:02.085521 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.012s	sys 0.027s
W20260709 12:13:02.102964 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 67001.1us
I20260709 12:13:02.095398 24654 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:02.107997 23751 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:02.120497 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:02.103073 24594 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:02.103228 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.128156 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.128271 23751 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.129614 24599 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.126125 24515 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:13:02.102861 24586 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.142350 23926 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.144881 24600 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.137146 23976 ts_tablet_manager.cc:1434] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.174s	user 0.008s	sys 0.000s
I20260709 12:13:02.141374 23782 raft_consensus.cc:2376] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:02.141781 24652 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.142325 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.142483 24577 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.143301 23934 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:02.143920 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.148948 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:02.144099 23930 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.127002 24598 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:13:02.146488 23976 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:13:02.149551 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:02.150048 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.155660 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:02.136169 24428 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:02.150484 23856 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.147413 23780 raft_consensus.cc:2376] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:02.152956 23785 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:02.156719 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.147544 23784 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:02.146456 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:02.124944 24632 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:13:02.145027 24586 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.162194 24577 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.165297 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:02.157449 24515 consensus_queue.cc:1243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:02.168565 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.181231 23784 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:02.160183 24506 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:02.175917 24495 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.156229 24581 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:13:02.176601 24493 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.193574 24543 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:13:02.193660 24493 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.194831 23782 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.195115 24543 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.177330 23857 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.167331 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:02.154460 24598 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.180609 23785 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:02.198506 23785 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:13:02.198838 23785 pending_rounds.cc:85] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 0
I20260709 12:13:02.199064 23785 pending_rounds.cc:107] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:13:02.199286 23785 raft_consensus.cc:2889] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:02.200708 23785 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.196143 24543 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.186470 23779 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:02.147130 24600 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:13:02.162787 23778 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.158958 24589 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:13:02.318544 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:02.339282 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 242981us
I20260709 12:13:02.418373 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.202695 24543 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.306639 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:02.310770 24489 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.181823 24428 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:02.345413 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.408551 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.196645 24515 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.219s
I20260709 12:13:02.428767 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.169395 24580 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:13:02.196151 24493 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.203420 24598 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:02.316936 23930 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.294808 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:02.312675 24628 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.197196 23857 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.411279 23779 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.430541 24581 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:02.338567 24652 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:02.368183 23783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.014s	sys 0.011s
W20260709 12:13:02.211364 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.196529 24506 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:02.420390 23899 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.405279 24495 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.328267 23975 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: No bootstrap required, opened a new log
I20260709 12:13:02.444360 23975 ts_tablet_manager.cc:1403] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent bootstrapping tablet: real 1.005s	user 0.024s	sys 0.035s
I20260709 12:13:02.432580 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 362) took 1135 ms. Trace:
W20260709 12:13:02.435750 23749 leader_election.cc:343] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:02.439833 23779 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:02.196645 23782 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.421602 24594 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:13:02.422353 24590 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 12:13:02.431907 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 310324us
I20260709 12:13:02.432806 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 351) took 1196 ms. Trace:
W20260709 12:13:02.445894 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 112091us
I20260709 12:13:02.446038 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.435384 24589 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:02.449268 23779 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.450038 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.461797 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:02.443717 23783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.014s	sys 0.011s
I20260709 12:13:02.464684 24428 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.465659 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 315) took 1353 ms. Trace:
I20260709 12:13:02.465368 24607 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.445978 24652 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.466022 23783 rpcz_store.cc:276] 0709 12:13:01.112742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.114855 (+  2113us) service_pool.cc:224] Handling call
0709 12:13:01.365694 (+250839us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:13:01.365860 (+   166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.366065 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:01.615198 (+249133us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.821893 (+206695us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b1800323a40. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:13:02.465317 (+643424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.465376 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:02.465420 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.465483 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.465604 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":257329,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":937,"spinlock_wait_cycles":84973056,"thread_start_us":891852,"threads_started":2,"wal-append.queue_time_us":821944}
I20260709 12:13:02.463714 24493 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.439750 24628 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.468330 23783 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.426177 24543 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.469241 24543 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.466369 24600 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.471197 23785 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:02.471431 23859 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
W20260709 12:13:02.463264 23825 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.952s (SENT)
I20260709 12:13:02.441425 24581 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:02.473443 24667 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:13:02.474166 24661 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:13:02.462992 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:02.451967 24660 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:13:02.463999 23780 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:02.463253 24657 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.465309 24577 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.476548 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.479228 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:02.521612 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45107.5us
I20260709 12:13:02.522001 23749 leader_election.cc:400] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:02.478933 24581 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.522622 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.522804 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:02.498183 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.475503 24491 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.481285 23778 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.449252 24495 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.502554 24506 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.504137 23859 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.508553 23784 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.513491 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:02.533157 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:02.534339 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:02.517206 24665 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:13:02.517683 24639 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.518580 24652 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:02.526393 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.475060 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.483943 24600 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.536372 23929 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.541343 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:02.482568 24493 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:13:02.450909 23925 rpcz_store.cc:276] 0709 12:13:01.234317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.653448 (+419131us) service_pool.cc:224] Handling call
0709 12:13:01.786598 (+133150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.787268 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.422646 (+635378us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54002a4750. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:02.430855 (+  8209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305339648}
I20260709 12:13:02.532239 24489 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:02.521615 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:02.542418 24576 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:02.543404 23932 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:02.499361 24657 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:02.522222 24607 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:02.478049 23857 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.552515 24652 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.559891 23857 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.560261 24590 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.560437 23779 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.561371 23858 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.561650 24575 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.549036 24491 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.562044 24575 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:02.562461 24491 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.564265 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 361) took 1288 ms. Trace:
I20260709 12:13:02.564648 23928 rpcz_store.cc:276] 0709 12:13:01.289913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.788380 (+498467us) service_pool.cc:224] Handling call
0709 12:13:01.788731 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.789089 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.789124 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.789580 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.158344 (+368764us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:13:02.158389 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.159811 (+  1422us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.564058 (+404247us) raft_consensus.cc:1685] finished
0709 12:13:02.564084 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.564207 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":972,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":362110,"threads_started":1,"wal-append.queue_time_us":542165}
I20260709 12:13:02.543452 23975 raft_consensus.cc:359] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:02.547652 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 84401.7us
I20260709 12:13:02.519299 23854 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:02.525731 24618 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:02.525321 24592 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.551817 23929 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.526074 24674 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.562913 24575 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.544068 24589 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:02.660003 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.805881 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.671862 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.771679 23785 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.806749 23975 raft_consensus.cc:385] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:02.648000 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:02.741587 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.833563 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:02.653061 23681 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:02.812412 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 287710us
I20260709 12:13:02.447073 23933 rpcz_store.cc:276] 0709 12:13:01.294927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.789560 (+494633us) service_pool.cc:224] Handling call
0709 12:13:01.789865 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.798006 (+  8141us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:13:01.896817 (+ 98811us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.413670 (+516853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.415404 (+  1734us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:02.421135 (+  5731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.424493 (+  3358us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.430274 (+  5781us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94759,"prepare.queue_time_us":99481,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4936,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5595,"thread_start_us":453640,"threads_started":1,"wal-append.queue_time_us":539756}
I20260709 12:13:02.794004 23976 tablet_bootstrap.cc:654] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
W20260709 12:13:02.744165 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.835170 23681 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:02.836277 23855 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:02.839949 23681 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:02.840796 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.841521 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.657608 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.656690 24489 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.842882 24489 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:02.843917 24489 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.845355 24489 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.842389 23899 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.848613 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:02.556232 24576 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:02.852118 24576 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.853511 24576 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:02.855686 23975 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Initialized, Role: FOLLOWER
I20260709 12:13:02.859179 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.657194 24521 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:02.860301 24521 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:02.861299 24521 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.862808 24521 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.842155 23855 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:13:02.869028 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.657117 23858 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:02.641224 23927 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:02.873936 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:02.645028 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:02.556453 24594 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.320s
I20260709 12:13:02.876209 23852 raft_consensus.cc:2376] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3. Re-sending same reply.
I20260709 12:13:02.800089 24682 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:13:02.878054 24682 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.654517 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.654224 23853 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:02.658974 24493 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.217s
W20260709 12:13:02.643754 24654 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.004s
I20260709 12:13:02.641637 23926 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:02.658010 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:02.567699 24639 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:02.841746 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:02.645618 24636 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.634521 23928 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:02.886747 23825 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.723311 24600 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.620739 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 350) took 1386 ms. Trace:
I20260709 12:13:02.864157 24575 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:02.644204 24580 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.241s
I20260709 12:13:02.721365 23857 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:02.642390 24495 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:02.927997 24495 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.771700 24577 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:02.898288 23975 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:02.836895 23854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 280) took 1598 ms (client timeout 1639 ms). Trace:
I20260709 12:13:02.636258 24628 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:02.888269 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.891642 23927 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:02.797220 24598 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.729086 24585 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.169s
I20260709 12:13:02.832777 24590 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.892048 24599 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:13:02.932463 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 406098us
I20260709 12:13:02.883209 24607 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.832360 24592 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.939615 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.880033 24618 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 12:13:02.945123 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 266602us
I20260709 12:13:02.640072 24657 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:02.947376 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:02.952895 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66233.4us
W20260709 12:13:02.953294 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.953835 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.937964 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 292) took 1033 ms. Trace:
I20260709 12:13:02.943117 23975 ts_tablet_manager.cc:1434] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent starting tablet: real 0.498s	user 0.008s	sys 0.015s
I20260709 12:13:02.922992 24594 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.900903 24636 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:02.979787 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:02.945863 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:02.999213 23900 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:03.003082 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49299.3us
I20260709 12:13:03.004050 23825 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:03.005193 23825 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:03.006726 23825 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.948673 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 370) took 1048 ms. Trace:
I20260709 12:13:02.937348 24581 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:02.954928 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 291299us
I20260709 12:13:03.022691 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:03.023247 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.023757 24577 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.024899 24577 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:03.026382 24577 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.027791 24600 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:03.028393 24600 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:03.029304 24600 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:03.031782 24589 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:03.032263 24589 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:03.033092 24589 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.034300 24589 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:03.036619 23927 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:13:02.948534 24631 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:13:03.012445 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:02.947976 24674 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:03.000049 23975 ts_tablet_manager.cc:1467] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Tablet startup took 555ms
I20260709 12:13:03.018738 24630 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.003880 24594 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:03.048911 24677 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:13:02.900054 23931 rpcz_store.cc:276] 0709 12:13:01.234088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.651217 (+417129us) service_pool.cc:224] Handling call
0709 12:13:01.651514 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.668386 (+ 16872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.670456 (+  2070us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.752716 (+ 82260us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.197692 (+444976us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:02.349096 (+151404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.410055 (+ 60959us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.549583 (+139528us) raft_consensus.cc:1685] finished
0709 12:13:02.552734 (+  3151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.567614 (+ 14880us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5522,"raft.queue_time_us":14882,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":1282,"thread_start_us":404388,"threads_started":1,"wal-append.queue_time_us":548308}
I20260709 12:13:03.010926 24628 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.049873 24543 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:02.997512 24495 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:02.956954 24632 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:02.957530 24599 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:03.060253 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47905.2us
W20260709 12:13:02.931835 23854 rpcz_store.cc:269] 0709 12:13:01.238015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.244846 (+  6831us) service_pool.cc:224] Handling call
0709 12:13:02.295367 (+1050521us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:02.832259 (+536892us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19318144}
I20260709 12:13:03.003482 24657 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:02.985054 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.010426 24621 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.039733 24428 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.020481 24607 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.051275 23930 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:03.071622 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:03.070158 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 227817us
I20260709 12:13:03.070364 24495 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.049062 24515 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:13:03.063704 24652 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:03.051103 23975 ts_tablet_manager.cc:1469] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Trace:
0709 12:12:44.322031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.344048 (+ 22017us) service_pool.cc:224] Handling call
0709 12:12:44.345782 (+  1734us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:44.345892 (+   110us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:44.426608 (+ 80716us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:44.474984 (+ 48376us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:44.475489 (+   505us) inbound_call.cc:177] Queueing success response
0709 12:13:01.372163 (+16896674us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:02.444685 (+1072522us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:02.541544 (+ 96859us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:02.542071 (+   527us) tablet_replica.cc:280] Starting consensus
0709 12:13:02.897973 (+355902us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b4c0028bfc0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aba7ea20 00007fb2aba8fa29 00007fb2aba55644 00007fb2aba7cf2d 00007fb2aba7cb93 00007fb2ad7ad18b 00007fb2ad7adcb5 00007fb2ad7a8f30 00007fb2ad7a7c40 00007fb2b11a4b2e 00007fb2b27c6d8a
Metrics: {"raft.queue_time_us":2055,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":5237,"spinlock_wait_cycles":20764544,"tablet-open.queue_time_us":16896767}
I20260709 12:13:03.010120 23857 rpcz_store.cc:276] 0709 12:13:01.897580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.017112 (+119532us) service_pool.cc:224] Handling call
0709 12:13:02.018561 (+  1449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:02.177189 (+158628us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b54000913a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad729dc0 00007fb2ad7b41c0 00007fb2ad7b5231 00007fb2ad7b1d8f 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c
0709 12:13:02.930774 (+753585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.945814 (+ 15040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17135360}
I20260709 12:13:03.081621 24577 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.081434 24628 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.088196 24689 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:03.094115 24491 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:03.094870 24590 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:03.095104 24607 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:13:03.095681 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.096135 24607 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:03.096498 24607 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:03.097295 24607 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.098603 24607 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:03.000159 23781 rpcz_store.cc:276] 0709 12:13:01.897841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.904408 (+  6567us) service_pool.cc:224] Handling call
0709 12:13:02.657433 (+753025us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:02.931555 (+274122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5521280}
W20260709 12:13:03.096150 23899 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.509s (SENT)
I20260709 12:13:03.102612 23899 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:03.083961 24630 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:03.083684 24695 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:13:03.092955 24680 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:13:03.079025 24495 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:03.115161 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:03.164070 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67925.9us
W20260709 12:13:03.164809 23899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.906s (SENT) [suppressed 94 similar messages]
W20260709 12:13:03.164889 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.087512 24687 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.178061 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62989.9us
I20260709 12:13:03.094130 24689 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:03.183921 24683 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:13:03.185171 24506 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.186537 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 312) took 1026 ms. Trace:
I20260709 12:13:03.086067 24493 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:13:03.081576 24428 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.092897 24636 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:03.119129 24695 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.093372 24577 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.093729 24628 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:03.196897 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 251118us
I20260709 12:13:03.182636 24652 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.183353 24630 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:03.187417 24621 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:13:03.203230 23749 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:03.215073 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 212768us
I20260709 12:13:03.188400 24689 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:03.187974 24592 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:03.223466 23856 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:03.187582 24586 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:13:03.202813 24543 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.231771 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:03.234447 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 189584us
I20260709 12:13:03.187393 24687 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.203387 24598 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:03.239089 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74303.7us
I20260709 12:13:03.196507 24695 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:03.220247 24621 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.246730 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:03.190922 23851 rpcz_store.cc:276] 0709 12:13:02.160207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.504651 (+344444us) service_pool.cc:224] Handling call
0709 12:13:02.505019 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.505436 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.505464 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.505997 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.863909 (+357912us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:02.875682 (+ 11773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.875757 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:03.186299 (+310542us) raft_consensus.cc:1685] finished
0709 12:13:03.186317 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:03.186471 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":301653,"threads_started":1,"wal-append.queue_time_us":385383}
W20260709 12:13:03.252344 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:03.263302 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.192865 24506 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.233352 24515 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.269533 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:03.270634 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:03.249504 24674 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.272149 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:03.275486 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:03.275859 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:03.284466 23860 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:03.276367 23899 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.972s (SENT)
I20260709 12:13:03.276687 24586 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.281250 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 306) took 1115 ms. Trace:
I20260709 12:13:03.245297 24581 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:03.265518 24621 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:03.225590 24652 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.309659 23856 rpcz_store.cc:276] 0709 12:13:02.150467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.160239 (+  9772us) service_pool.cc:224] Handling call
0709 12:13:02.160480 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.225287 (+1064807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.265828 (+ 40541us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003cc654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:03.266008 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2443760896}
I20260709 12:13:03.309612 24630 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.194685 24599 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.242328 24543 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.244179 24592 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:13:03.237157 24600 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:03.316265 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:03.333563 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.397497 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:03.470185 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72805us
I20260709 12:13:03.286764 24598 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:03.316452 24686 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 12:13:03.472098 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:03.422330 24673 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:13:03.297184 24703 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.365665 24590 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.384586 24632 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.419379 23931 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:03.281250 24702 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:03.568434 24685 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.349s	user 0.000s	sys 0.000s
I20260709 12:13:03.442060 24696 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:03.566411 24590 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.309491 24699 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.279s
I20260709 12:13:03.565290 24708 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.283639 24515 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.395337 24593 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:03.287366 24576 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:03.315773 24636 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.385288 24618 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:03.420645 24693 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.366271 24692 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:13:03.471717 24521 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.000s
I20260709 12:13:03.341027 24704 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.564124 24703 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.312158 24586 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.582695 24708 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.340744 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:04.401389 23776 consensus_meta.cc:287] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.465s	user 0.010s	sys 0.010s
I20260709 12:13:03.587766 24506 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.401384 24581 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:13:04.091827 23860 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.031s	sys 0.069s
W20260709 12:13:03.787175 24657 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.000s
I20260709 12:13:03.946111 24621 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:03.775995 24667 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:13:04.413514 23860 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.039s	sys 0.101s
W20260709 12:13:03.829872 24711 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:13:04.102488 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.015s	sys 0.048s
I20260709 12:13:04.422307 24598 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:04.424216 24628 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.423375 24598 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.425338 23976 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:13:04.426281 23976 ts_tablet_manager.cc:1403] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 2.280s	user 0.049s	sys 0.057s
I20260709 12:13:04.408710 24702 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:04.427874 24598 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.427737 24702 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.087110 23826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.002s	sys 0.006s
W20260709 12:13:04.431869 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.026s	sys 0.061s
I20260709 12:13:04.430414 24586 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.085091 23858 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 12:13:04.057416 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.015s	sys 0.017s
W20260709 12:13:04.435532 23826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.004s	sys 0.016s
W20260709 12:13:04.435935 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.025s	sys 0.040s
W20260709 12:13:04.081821 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.048718 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.442243 24600 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.560168 24489 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:03.887894 24682 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.009s	user 0.008s	sys 0.003s
W20260709 12:13:03.576298 24709 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:13:04.409965 23776 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.435184 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 390) took 1145 ms. Trace:
I20260709 12:13:03.561226 24585 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.435825 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 391) took 1159 ms. Trace:
W20260709 12:13:03.781402 24689 consensus_meta.cc:287] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.555s	user 0.000s	sys 0.008s
W20260709 12:13:03.591471 24575 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.668s	user 0.006s	sys 0.003s
I20260709 12:13:04.454368 24684 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:03.589687 24654 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:03.764968 24705 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.450s	user 0.000s	sys 0.002s
I20260709 12:13:03.728860 24643 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:13:03.783681 24589 consensus_meta.cc:287] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.749s	user 0.011s	sys 0.001s
W20260709 12:13:03.889880 24607 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.000s
I20260709 12:13:04.446859 24598 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.587551 24631 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.724697 24493 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.723642 24577 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.461027 23858 rpcz_store.cc:276] 0709 12:13:03.276056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.314897 (+ 38841us) service_pool.cc:224] Handling call
0709 12:13:04.084283 (+769386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.433663 (+349380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.435764 (+  2101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.465925 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 387) took 1207 ms. Trace:
I20260709 12:13:03.586144 24687 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.471310 24580 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:04.496512 24580 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:04.498798 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 375) took 2305 ms. Trace:
I20260709 12:13:04.492053 23931 rpcz_store.cc:276] 0709 12:13:03.238002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.241668 (+  3666us) service_pool.cc:224] Handling call
0709 12:13:03.286886 (+ 45218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.566336 (+279450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.435744 (+869408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.500308 24576 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:04.499248 23934 rpcz_store.cc:276] 0709 12:13:02.193460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.193603 (+   143us) service_pool.cc:224] Handling call
0709 12:13:02.193970 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.307416 (+113446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.311925 (+  4509us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.646378 (+334453us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b2c000bb9c8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:13:02.646834 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.002717 (+355883us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:03.009847 (+  7130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.498208 (+1488361us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b54002030d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:04.498304 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.498373 (+    69us) raft_consensus.cc:1685] finished
0709 12:13:04.498390 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.498645 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":966,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":9092,"spinlock_wait_cycles":10107520,"thread_start_us":309127,"threads_started":1,"wal-append.queue_time_us":437624}
I20260709 12:13:04.500870 24576 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:03.712393 24593 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.446379 24693 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.466426 24600 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.491935 24586 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:04.510500 24714 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.487757 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.13827e+06us
I20260709 12:13:04.457494 23860 rpcz_store.cc:276] 0709 12:13:03.275813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.285312 (+  9499us) service_pool.cc:224] Handling call
0709 12:13:03.285609 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.285789 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:03.560749 (+274960us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.415123 (+854374us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.415328 (+   205us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:04.415469 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.416321 (+   852us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.421274 (+  4953us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":400739,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":864048,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":245,"thread_start_us":1128382,"threads_started":2,"wal-append.queue_time_us":17046}
W20260709 12:13:04.503464 24491 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.004s	sys 0.003s
I20260709 12:13:04.510872 24702 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:03.584149 24632 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:03.600612 24696 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.510432 24674 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.526043 24491 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.004s	sys 0.003s
I20260709 12:13:04.518646 24654 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.531248 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 271) took 2287 ms. Trace:
W20260709 12:13:04.520754 24495 consensus_meta.cc:287] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.333s	user 0.008s	sys 0.004s
W20260709 12:13:04.534673 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.532783 24696 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.532461 24714 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.519639 24643 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:04.508673 24598 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.520411 24621 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.531476 24586 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.520244 24600 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.531828 24704 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.515514 24489 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.533244 24703 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.540637 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 382) took 1260 ms. Trace:
I20260709 12:13:04.518502 24684 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.543668 24495 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.544608 24674 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.544538 24493 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.533989 23932 rpcz_store.cc:276] 0709 12:13:02.204500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.204671 (+   171us) service_pool.cc:224] Handling call
0709 12:13:02.543089 (+338418us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:13:02.543254 (+   165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.746986 (+203732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.747020 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.747515 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.001971 (+254456us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:03.070803 (+ 68832us) log.cc:844] Serialized 14 byte log entry
0709 12:13:03.071302 (+   499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.071450 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.463518 (+1392068us) raft_consensus.cc:1685] finished
0709 12:13:04.483164 (+ 19646us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.491717 (+  8553us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181415,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":154026,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":161,"spinlock_wait_cycles":2325248,"thread_start_us":317556,"threads_started":2,"wal-append.queue_time_us":438604}
I20260709 12:13:04.547909 24654 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.549947 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 383) took 1275 ms. Trace:
I20260709 12:13:04.550596 24708 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.549849 24714 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.550534 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 385) took 1275 ms. Trace:
I20260709 12:13:04.551455 24723 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.539876 24702 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.606529 24702 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.606055 24723 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.552497 24577 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.609706 24577 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.610477 24723 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.604310 23854 rpcz_store.cc:276] 0709 12:13:03.274716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.281325 (+  6609us) service_pool.cc:224] Handling call
0709 12:13:03.281702 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.310645 (+ 28943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.314502 (+  3857us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.473875 (+159373us) log.cc:844] Serialized 26 byte log entry
0709 12:13:04.088076 (+614201us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:04.088190 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.088360 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.550080 (+461720us) raft_consensus.cc:1685] finished
0709 12:13:04.550113 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.550327 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":385,"raft.queue_time_us":34642,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":79520,"thread_start_us":521093,"threads_started":1,"wal-append.queue_time_us":965458}
I20260709 12:13:04.604202 24689 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.553797 24657 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.555202 24643 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.557526 23976 raft_consensus.cc:359] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.558521 23777 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.558763 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:04.559020 24585 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.562973 24696 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:04.563866 24713 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:04.565379 24652 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.567432 24726 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.533221 24632 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.569749 23859 rpcz_store.cc:276] 0709 12:13:03.273930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.274075 (+   145us) service_pool.cc:224] Handling call
0709 12:13:03.274379 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.276937 (+  2558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.276971 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.287880 (+ 10909us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.475320 (+187440us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:03.558726 (+ 83406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.558812 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:03.829289 (+270477us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:04.520502 (+691213us) raft_consensus.cc:1685] finished
0709 12:13:04.520549 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.534197 (+ 13648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52168,"raft.queue_time_us":7581,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":23422,"spinlock_wait_cycles":26281856,"thread_start_us":187101,"threads_started":1,"wal-append.queue_time_us":417462}
I20260709 12:13:04.585912 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 384) took 1311 ms. Trace:
I20260709 12:13:04.588881 23852 rpcz_store.cc:276] 0709 12:13:03.274201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.282306 (+  8105us) service_pool.cc:224] Handling call
0709 12:13:04.075754 (+793448us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:13:04.076276 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:04.077114 (+   838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.077174 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:04.078197 (+  1023us) log.cc:844] Serialized 26 byte log entry
0709 12:13:04.434810 (+356613us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:04.440144 (+  5334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.447209 (+  7065us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.549388 (+102179us) raft_consensus.cc:1685] finished
0709 12:13:04.549417 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.549723 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":348709,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":172,"spinlock_wait_cycles":7174912,"thread_start_us":340929,"threads_started":1,"wal-append.queue_time_us":430025}
I20260709 12:13:04.594813 24495 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.594903 24493 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.596345 24586 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.598448 24684 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.600134 24704 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.552170 24693 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.552109 24725 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.691574 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.692504 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.691967 24725 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.693732 24631 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.694702 23777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 308) took 2609 ms (client timeout 1581 ms). Trace:
I20260709 12:13:04.696036 24708 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.696197 24631 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.696762 24631 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:04.697022 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.553172 24575 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:04.613317 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.612035 24577 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.644908 23976 raft_consensus.cc:385] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:04.609175 24702 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.582655 23853 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.581918 24682 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:04.696633 24708 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.586701 24489 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:04.699659 23976 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
W20260709 12:13:04.695143 23777 rpcz_store.cc:269] 0709 12:13:02.085536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.093951 (+  8415us) service_pool.cc:224] Handling call
0709 12:13:02.654334 (+560383us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:04.694377 (+2040043us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54001efd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:04.694532 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3614198528}
I20260709 12:13:04.604453 24636 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.704236 24714 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.718058 24585 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:04.707743 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.718796 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 160204us
I20260709 12:13:04.704205 24652 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.735651 23976 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.716387 24674 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.727735 24643 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:04.713192 23926 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:04.718638 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 376) took 2054 ms. Trace:
I20260709 12:13:04.701002 24586 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.725744 24598 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.741430 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:04.741950 24725 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.739039 23925 rpcz_store.cc:276] 0709 12:13:02.664399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.701161 (+ 36762us) service_pool.cc:224] Handling call
0709 12:13:02.767174 (+ 66013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.931194 (+164020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.937858 (+  6664us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.002646 (+ 64788us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.246310 (+243664us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:13:03.246356 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.246429 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.718183 (+1471754us) raft_consensus.cc:1685] finished
0709 12:13:04.718236 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.718443 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108658,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"thread_start_us":329396,"threads_started":2,"wal-append.queue_time_us":369530}
I20260709 12:13:04.687287 23857 rpcz_store.cc:276] 0709 12:13:03.274460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.280429 (+  5969us) service_pool.cc:224] Handling call
0709 12:13:03.280782 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:04.442870 (+1162088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.442917 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:04.443797 (+   880us) log.cc:844] Serialized 26 byte log entry
0709 12:13:04.495061 (+ 51264us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:13:04.496901 (+  1840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.553306 (+ 56405us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400386b54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:04.553361 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.585416 (+ 32055us) raft_consensus.cc:1685] finished
0709 12:13:04.585447 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.585734 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1755,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":166,"spinlock_wait_cycles":2644697344,"thread_start_us":39963,"threads_started":1,"wal-append.queue_time_us":122704}
I20260709 12:13:04.745340 24636 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.700533 24575 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.745800 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.747745 24631 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.748956 24643 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:04.749178 24723 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.749435 24575 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.713395 24657 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.719081 23853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 386) took 1444 ms (client timeout 1892 ms). Trace:
W20260709 12:13:04.750373 23853 rpcz_store.cc:269] 0709 12:13:03.274905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.282255 (+  7350us) service_pool.cc:224] Handling call
0709 12:13:04.706891 (+1424636us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.751067 24585 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.712522 24704 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:04.751722 24704 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:04.751811 24657 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.752107 24493 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.746737 24586 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.707614 24696 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.751770 24585 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.752820 24493 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.756861 24586 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.739111 24652 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.749935 24631 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.756156 24575 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.755977 24714 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.717491 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:04.752194 24704 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:04.730787 24689 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.707096 24726 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.714238 24589 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.772377 24589 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.772893 24576 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.774613 24589 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.774986 24576 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.756896 24585 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.767510 23929 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
W20260709 12:13:04.779745 24695 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.532s	user 0.005s	sys 0.004s
I20260709 12:13:04.780057 24682 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.759166 24636 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.768345 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:04.780797 24682 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.782346 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:04.783617 24682 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.768571 23856 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:04.768002 23853 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:04.769340 24493 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.718314 24713 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.789839 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.758122 24657 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.770248 24652 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.770891 24708 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.771730 24696 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.749776 24723 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.776963 24543 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.780180 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 382) took 2077 ms. Trace:
I20260709 12:13:04.768136 24575 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.806020 24575 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.780839 24636 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.808816 24636 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.789024 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.728842 24593 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.794911 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.809397 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.810186 23851 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:04.810909 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.795480 23855 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:04.801007 23929 rpcz_store.cc:276] 0709 12:13:02.702918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.887049 (+184131us) service_pool.cc:224] Handling call
0709 12:13:02.887306 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.779105 (+1891799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.779863 (+   758us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540006b3d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:04.780105 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4315444096}
I20260709 12:13:04.813036 24489 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.796059 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.817724 24515 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.817958 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.818739 23855 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:04.821496 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 386) took 1585 ms. Trace:
I20260709 12:13:04.822009 23930 rpcz_store.cc:276] 0709 12:13:03.235821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.241723 (+  5902us) service_pool.cc:224] Handling call
0709 12:13:04.422223 (+1180500us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:13:04.422667 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.597150 (+174483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.597194 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.597281 (+    87us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:04.597341 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.821060 (+223719us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54000f78d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:04.821118 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.821304 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403172096}
W20260709 12:13:04.824112 23776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 288) took 2927 ms (client timeout 1906 ms). Trace:
I20260709 12:13:04.824388 24714 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.825073 24714 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:04.824137 24594 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.825913 24515 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.825697 24714 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.826380 24515 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:04.794755 24657 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.796486 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.796542 24589 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:04.788156 24725 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:04.837355 24725 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:04.838703 24725 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.839347 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.809448 24723 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.839918 24515 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:04.776963 24726 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.752743 24643 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.797994 24713 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.811004 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.857147 24643 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.857574 23851 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:04.859508 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.860265 23851 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:04.795887 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:04.824541 23776 rpcz_store.cc:269] 0709 12:13:01.896823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.904330 (+  7507us) service_pool.cc:224] Handling call
0709 12:13:02.657869 (+753539us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:04.823892 (+2166023us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b54001ef854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:04.824019 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86856832}
I20260709 12:13:04.759327 24632 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.829564 24428 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.864156 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.865192 23851 raft_consensus.cc:2410] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:13:04.845468 24590 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.829725 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.847954 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.866305 23857 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:04.836632 24589 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:04.867015 24589 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:04.866746 24632 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.867772 23853 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:04.868902 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:04.870154 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:04.852047 23899 connection.cc:256] RPC call timeout handler was delayed by -1.1920787560000008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
I20260709 12:13:04.853312 24657 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.847848 23859 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:04.830323 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.863193 24643 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.848932 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:04.866192 23852 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.792606 24631 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:04.874140 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.871315 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:04.815514 24489 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.866168 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:04.877189 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.879331 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.880983 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.881301 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.881666 24632 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.881711 23859 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:04.878914 24428 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.882622 24428 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.884212 24657 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:04.884479 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 84882.5us
I20260709 12:13:04.887030 24428 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.887154 23851 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:04.896256 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:04.875990 24643 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.897442 24657 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:04.898098 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:04.898702 24643 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.898941 24657 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:04.873909 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.893596 23976 ts_tablet_manager.cc:1434] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.467s	user 0.012s	sys 0.004s
I20260709 12:13:04.899931 23857 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:04.901337 23976 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:13:04.780223 24695 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.887985 23852 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
W20260709 12:13:04.902529 23899 connection.cc:256] RPC call timeout handler was delayed by -1.0484978119998232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 12:13:04.903332 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45769.6us
I20260709 12:13:04.904196 24713 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:13:04.907657 24714 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.912206 23856 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:04.916311 24515 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.878566 24631 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.912533 24695 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.923775 24657 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.924415 24657 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.926829 24657 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.927114 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.929061 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.930238 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.930828 23856 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:04.931103 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.920665 24515 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.931694 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.931823 24689 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.932487 24689 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.932919 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.931761 23858 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.934293 23856 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.936373 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.919895 24489 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.936679 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.938988 23853 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.939435 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.940111 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.937768 24689 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.936081 23749 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:04.918174 24704 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.940124 23852 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.945410 24592 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.946079 24592 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.948583 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:04.949270 23856 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:04.954075 24592 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.954351 24704 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.933773 24515 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:04.955917 24592 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.963110 23854 raft_consensus.cc:2410] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:04.963312 24592 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:04.965987 24592 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:04.977829 24589 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:04.982731 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.985438 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 106) took 2865 ms. Trace:
W20260709 12:13:04.986148 23825 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.601s (SENT)
I20260709 12:13:04.960044 24657 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:04.986611 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:04.987607 23899 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.588s (SENT)
I20260709 12:13:04.987987 24643 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:04.988057 23899 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:04.989022 24592 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:04.989382 23899 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.590s (SENT)
I20260709 12:13:04.989863 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:04.990625 24592 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:05.002449 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.004413 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.006980 24643 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:05.007567 24594 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.008285 24594 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.008661 24594 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.009989 24631 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.010888 24589 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.011914 24580 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.013725 24580 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:05.017513 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 81585.1us
I20260709 12:13:05.022806 24618 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.023334 24618 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.023722 24618 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:05.014482 23825 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.738s (SENT)
I20260709 12:13:05.026975 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:04.994974 24489 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:05.038005 24489 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:05.038862 23900 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.040263 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.041448 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.042173 24696 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:05.042464 23899 leader_election.cc:336] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.757s (SENT)
I20260709 12:13:05.042611 24696 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.042912 23899 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:05.042990 24696 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.042086 24592 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.044468 24590 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.044584 24592 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.044889 24590 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.045099 24592 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:05.044129 23899 leader_election.cc:336] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.759s (SENT)
I20260709 12:13:05.047895 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:05.049913 23900 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.059878 23858 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:05.057196 24590 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.063359 23900 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.061741 24631 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:05.069182 24585 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.065963 24682 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:05.069656 24585 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.060401 24636 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:05.069973 24682 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:05.070470 24682 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:05.070021 24585 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:04.985862 23681 rpcz_store.cc:276] 0709 12:13:02.119684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.127761 (+  8077us) service_pool.cc:224] Handling call
0709 12:13:04.985203 (+2857442us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:03.194440 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:03.397743 (+203303us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:03.419878 (+ 22135us) write_op.cc:435] Acquired schema lock
0709 12:13:03.421423 (+  1545us) tablet.cc:662] Decoding operations
0709 12:13:03.777171 (+355748us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:03.777249 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:03.777285 (+    36us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:13:03.777706 (+   421us) tablet.cc:701] Row locks acquired
0709 12:13:03.777720 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:13:03.777888 (+   168us) write_op.cc:270] Start()
0709 12:13:03.778012 (+   124us) write_op.cc:276] Timestamp: P: 1783599183777834 usec, L: 0
0709 12:13:03.778030 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:13:03.778810 (+   780us) log.cc:844] Serialized 1693 byte log entry
0709 12:13:03.856376 (+ 77566us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2aa6a80ac 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b
0709 12:13:04.793663 (+937287us) op_driver.cc:464] REPLICATION: finished
0709 12:13:04.926180 (+132517us) write_op.cc:301] APPLY: starting
0709 12:13:04.926279 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:04.926461 (+   182us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:04.926493 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:04.927652 (+  1159us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:04.927924 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:13:04.927988 (+    64us) write_op.cc:312] APPLY: finished
0709 12:13:04.928337 (+   349us) log.cc:844] Serialized 40 byte log entry
0709 12:13:04.929126 (+   789us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:04.929342 (+   216us) write_op.cc:454] Released schema lock
0709 12:13:04.929409 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:04.929838 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:04.929864 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":123482,"apply.run_cpu_time_us":2628,"apply.run_wall_time_us":3659,"num_ops":4,"prepare.queue_time_us":229112,"prepare.run_cpu_time_us":13864,"prepare.run_wall_time_us":826782,"raft.queue_time_us":32432,"raft.run_cpu_time_us":58309,"raft.run_wall_time_us":80436,"replication_time_us":1015490,"spinlock_wait_cycles":120805248,"thread_start_us":492641,"threads_started":4,"wal-append.queue_time_us":727631}]]}
I20260709 12:13:05.063736 24493 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:05.078323 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:05.085412 24589 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:05.087229 23749 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:05.090206 24594 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.088423 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:05.092361 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 87166.1us
I20260709 12:13:05.093536 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:05.101369 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.095292 24726 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.102381 24726 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.102382 24631 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:13:05.098932 24589 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:05.109153 23900 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.111263 24695 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:05.099774 24713 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.111724 24695 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:05.111919 24713 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.112397 24713 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.115985 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.116899 23858 raft_consensus.cc:2410] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:05.118592 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.119238 23855 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:05.105466 24726 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.121917 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:05.123070 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.123338 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:05.124800 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.125511 23858 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:05.126176 24695 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:05.124023 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.127169 23851 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:05.127496 24594 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.129465 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.130050 23860 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:05.133337 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46263.4us
I20260709 12:13:05.138718 23976 tablet_bootstrap.cc:654] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:05.096309 24618 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.141379 24682 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.142524 24618 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.128144 23683 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:05.126959 24592 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:05.145645 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.146608 24589 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:05.147336 24631 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:05.147816 24631 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:05.148293 24695 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.148912 24631 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.150288 24631 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:05.152460 24696 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:05.153560 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50375us
W20260709 12:13:05.155010 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.158896 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.160406 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.163748 23855 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:05.145829 23683 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.164846 24682 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.166115 24696 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.166385 24695 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.184244 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:05.184278 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.185252 23859 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.185364 24589 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:05.187218 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:05.188673 24589 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:13:05.190665 23749 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:05.192031 24589 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:05.192476 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.193238 23856 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.194144 23749 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:05.196573 24589 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:05.197367 24589 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:05.198374 24589 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.202597 23856 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.203943 24674 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:05.204643 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.207546 24674 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:05.212558 23859 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.216702 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:05.224324 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.227018 23853 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:05.231578 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.232313 23932 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:05.233608 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.234915 23859 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.249076 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.250128 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.251192 24491 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.251570 24491 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.251915 24491 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.253703 23852 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:05.255890 24489 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.256322 24489 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.256649 24489 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:05.241346 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 73251.5us
I20260709 12:13:05.251302 23900 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.274148 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.241766 24594 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:13:05.281294 23683 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:05.264873 24696 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:13:05.260740 24723 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:13:05.288321 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.288599 24493 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:05.289276 23683 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:05.289741 24691 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:05.290372 24657 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:05.286167 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.275573 24674 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.293325 23932 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:05.294633 23859 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.273111 24695 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:13:05.303670 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.304013 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.283712 24593 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:05.308228 23900 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.302333 23683 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:05.276794 24491 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.314903 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.315048 23683 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:05.323652 23683 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:05.315897 23933 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:05.331881 24489 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:05.337101 24696 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:05.337693 24696 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.338660 24696 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.339912 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.341895 24657 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:05.353067 24695 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:05.357563 24726 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.352977 24713 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.362776 24590 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.363235 24713 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.308903 24674 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.364110 24713 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:05.364665 24713 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.365885 24674 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.365712 24713 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.316587 24493 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.367770 24493 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.368871 24699 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:05.369468 24699 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.370404 24699 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.371730 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.373062 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.316419 24689 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:05.374023 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.319908 24593 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:05.386225 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 58925.5us
I20260709 12:13:05.384265 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.387919 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.389364 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.389807 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.389955 23856 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:05.377043 23751 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.393848 23931 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.389140 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.395606 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:05.389140 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.333706 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.398546 23932 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.400286 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.336376 24489 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:05.384320 24580 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.385195 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.404132 23751 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.406559 23858 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:05.409806 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.396847 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.381626 24674 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:05.418887 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.419716 23929 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.385705 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.402920 24657 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.420831 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.394017 24689 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.412503 24725 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.411075 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.403712 23854 raft_consensus.cc:2376] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1. Re-sending same reply.
I20260709 12:13:05.412817 24491 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.430073 23900 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:05.431680 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 108146us
I20260709 12:13:05.418793 24726 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.411850 23926 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:05.434971 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.409173 24580 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.438127 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.439718 23929 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.435911 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.423022 24657 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.413235 24585 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.426927 24716 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.428887 24725 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.445827 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.446089 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.443933 24639 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:13:05.433970 23933 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:05.456964 23930 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:05.448086 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.457592 23854 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.457994 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.458643 23930 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:05.448081 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.460640 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.461297 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.461728 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.462082 23976 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:13:05.462321 23934 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:05.462540 23976 ts_tablet_manager.cc:1403] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.562s	user 0.021s	sys 0.037s
I20260709 12:13:05.462117 23931 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.439504 24691 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:05.455621 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.442845 24704 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.442607 24657 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.470232 23929 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:05.452353 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.471226 23851 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.472223 24592 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.450774 24725 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.455940 23853 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.453301 24726 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:05.466665 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.476259 23930 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:05.476944 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.477535 24765 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.477742 23933 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.477959 24765 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:05.478329 24765 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.480741 24590 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.481540 23751 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.474850 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.432956 24593 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.468564 24704 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:05.491644 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.492209 24766 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.473759 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.503780 24704 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.492928 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.504063 24580 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.472783 24691 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.505311 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.505223 24691 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.506009 23927 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:05.506978 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.493952 24585 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:05.494560 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.496208 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.499905 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.463400 24716 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.502903 23853 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:05.508249 23927 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.469539 24674 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:05.515791 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.525298 23931 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:05.524096 24767 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.508181 23930 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:05.486568 24726 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:05.526818 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 53686.2us
I20260709 12:13:05.527269 24674 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.527608 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.528076 23751 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.515794 24766 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.534185 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.535768 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.539403 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.539412 24774 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.540355 24774 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.540948 24774 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.542145 23854 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.543402 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.545998 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.546667 23856 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:05.553825 24760 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260709 12:13:05.553731 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:05.532462 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.561674 23933 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
I20260709 12:13:05.563095 23933 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:13:05.563421 23933 pending_rounds.cc:85] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 1
I20260709 12:13:05.563756 23933 pending_rounds.cc:107] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:13:05.564038 23933 raft_consensus.cc:2889] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.534386 24766 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:05.561118 24768 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:13:05.545359 24762 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:05.574390 24762 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.531993 24767 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.575352 24762 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:05.577386 24767 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.577937 24767 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.578269 24767 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.578579 24767 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.554211 24592 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:05.555352 24589 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.537057 23925 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:05.544471 24772 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.579730 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.578860 24657 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:05.591715 24764 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:13:05.604878 23976 raft_consensus.cc:359] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.606366 23853 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.591877 24689 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.599702 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.588195 24775 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.608770 23976 raft_consensus.cc:385] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:05.609846 24592 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.630097 23976 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:13:05.602133 24674 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.604604 24774 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.630997 23856 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:05.634477 23899 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.906s (SENT)
I20260709 12:13:05.596204 24589 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.628409 24772 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.630030 24775 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.648049 24775 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.631425 23976 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.647214 24772 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.649523 23899 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:05.633008 23854 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.626989 24689 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:05.650238 23824 leader_election.cc:343] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:05.657380 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.669878 24589 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.670856 24704 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.618266 24726 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.674646 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:05.675530 23824 leader_election.cc:343] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:05.679610 24639 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.679378 24689 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.679785 24714 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.680238 24714 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.634891 24592 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.680557 24714 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.677767 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.672276 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.690268 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.707270 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.707948 23932 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:05.708942 23824 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:05.707326 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72900.4us
I20260709 12:13:05.678702 24585 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:05.672148 24716 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.705684 24631 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.713613 24543 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.682082 24515 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:13:05.686429 24639 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.715562 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.719460 23976 ts_tablet_manager.cc:1434] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.257s	user 0.005s	sys 0.004s
I20260709 12:13:05.723505 23976 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Bootstrap starting.
I20260709 12:13:05.725558 23824 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.720100 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.727551 23932 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:05.726501 24428 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.728750 24428 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.729081 24428 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.732327 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.678965 24725 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.735291 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.736258 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.711727 23930 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:05.747742 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.749567 24725 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.717278 24631 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.751367 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.718912 24543 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.726832 24639 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.734578 24643 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.758409 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.737316 24716 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:05.763924 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.764009 24515 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:05.765931 24774 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.766661 24682 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:05.779477 24515 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.767635 24543 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.779990 24515 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.770658 24643 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.781474 24643 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.769408 23932 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 1.
I20260709 12:13:05.784377 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.785640 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.786660 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.787189 23932 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:05.763180 24689 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.787477 24515 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.779836 24682 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:05.790060 24682 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.791278 24682 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:05.791810 24657 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.792568 24515 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.754537 24632 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:05.766978 24767 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.757364 24631 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.839747 24543 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.841034 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:05.843575 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.843791 23855 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:05.771369 24716 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:05.845191 23932 raft_consensus.cc:2410] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:13:05.805411 24632 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:05.863567 23855 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.787333 24765 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.849205 24716 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.860848 24632 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:05.857985 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:05.882627 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:05.905686 24543 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.883518 24696 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.916684 23854 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:05.949445 24776 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:13:05.954860 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 203537us
I20260709 12:13:05.903908 24704 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.955788 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:05.918895 23930 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:13:05.972316 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.962288 24543 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:13:05.989863 23976 tablet_bootstrap.cc:654] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Neither blocks nor log segments found. Creating new log.
I20260709 12:13:05.994014 24766 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.005693 24775 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.008292 24765 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:05.947734 24696 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:05.970677 23853 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:05.984887 23851 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:06.019340 24767 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:06.027808 24718 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 12:13:06.023252 24580 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:06.011706 24716 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.015928 24631 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.042726 24631 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.045322 24696 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:06.046752 24716 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:06.047147 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.048825 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.050997 24515 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:06.051376 24515 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.052138 24515 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.014992 24428 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:06.054134 24778 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:13:06.054443 24428 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:06.049047 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.015252 24643 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.057682 24643 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.016604 24714 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.060463 24714 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.060228 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:06.061158 23930 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:06.061357 24766 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.042955 24639 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.062592 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.064220 23931 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.069283 24713 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:06.070199 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:06.070798 23929 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.071673 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.072184 23929 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.074901 23853 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:06.078002 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.079465 24789 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:06.079685 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.080408 23930 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:13:06.081673 24785 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:13:06.086087 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 494938us
I20260709 12:13:06.086613 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:06.087986 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.088631 23776 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.090870 24515 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.092447 23777 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:06.094414 24580 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:06.094305 24632 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.094877 24580 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.095453 24639 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.095882 24775 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.100049 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.100638 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.102536 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.103225 23782 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.104079 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.100864 23778 raft_consensus.cc:2368] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:06.105654 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.105914 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.106654 23778 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:13:06.091495 23931 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.108624 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.079918 24789 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.098727 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.098922 23777 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:13:06.099272 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.105032 23785 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.116716 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.119510 23929 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.131079 23779 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.087299 23926 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:06.205595 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.210745 23931 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:06.211861 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 95257.5us
I20260709 12:13:06.150561 23751 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:06.200311 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.213143 23927 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:06.215036 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.218194 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:06.213265 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 62802.7us
I20260709 12:13:06.099973 23783 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:06.226541 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.224399 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.115818 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.204845 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.205437 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.231743 23932 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.107662 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.137414 24789 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.228986 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.235934 23855 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:06.225281 24639 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.238623 23778 raft_consensus.cc:2376] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2. Re-sending same reply.
I20260709 12:13:06.239346 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.239667 23926 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:06.239845 23857 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:06.221906 24783 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:06.220486 24643 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:06.242892 24696 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.243034 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.193389 23781 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:06.235809 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:06.238683 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.254400 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.254738 23858 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:06.254020 23933 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.207342 24782 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.258765 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.257588 23777 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.237443 24632 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.262476 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.266196 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.269073 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:06.269300 23900 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:06.257443 24793 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:13:06.267282 23751 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:06.257704 24723 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.274749 23931 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.276976 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.274919 23900 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.259852 24643 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.266629 24590 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.259716 24783 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.277354 23824 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.260324 24639 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:06.274084 24726 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.266811 24765 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:06.286525 24590 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:06.286787 24723 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:06.287701 23932 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:06.289710 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.290313 24543 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.290730 24428 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:06.283521 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.286068 24726 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:06.266484 24696 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:06.290441 23784 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.304244 24636 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:06.304876 24636 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:06.292018 24767 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:06.305783 24767 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:06.306706 24767 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.287726 24723 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.282723 24643 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.282087 24682 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:06.304234 24630 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:06.286494 24765 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.284341 23851 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:06.282276 24783 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.319766 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.268003 24782 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.324407 24543 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:06.344501 23784 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.346493 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.348094 24515 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.305809 24636 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.348810 24696 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.320822 24674 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.307343 24580 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.352979 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.314348 23777 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.320753 24704 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.319119 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.344552 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.321471 24630 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:06.367758 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 77643.8us
W20260709 12:13:06.358464 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71327.5us
I20260709 12:13:06.366535 23779 raft_consensus.cc:2376] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2. Re-sending same reply.
I20260709 12:13:06.370095 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:06.326725 24765 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.371976 23784 raft_consensus.cc:2368] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:06.354568 24682 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:06.376425 23781 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:06.374485 24674 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.378742 24674 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.379774 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.378626 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.381958 23860 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:06.382843 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.383695 23852 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:06.383392 24682 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.384665 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.385447 24515 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.385936 24515 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.387135 23855 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:06.387542 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.388206 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.388921 23855 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:06.388813 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.389765 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.390455 23855 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:06.391171 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.393504 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.394320 23858 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:06.395082 24580 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.396253 23851 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:06.396982 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.397351 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.398834 23851 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:06.381484 24590 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.400033 24726 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.379696 23782 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.402343 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.395243 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.404503 23858 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:06.396255 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:06.390074 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.356799 24715 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:13:06.356230 24639 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.372453 24630 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:06.375833 23776 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:06.441334 23933 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.412092 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.380378 24632 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:06.382863 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.375129 24704 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.472901 23784 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:06.473363 23899 leader_election.cc:336] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.576s (SENT)
I20260709 12:13:06.559090 23976 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: No bootstrap required, opened a new log
I20260709 12:13:06.584362 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:06.549491 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.516525 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.584573 23976 ts_tablet_manager.cc:1403] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Time spent bootstrapping tablet: real 0.861s	user 0.027s	sys 0.031s
I20260709 12:13:06.566529 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:06.592037 23934 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
W20260709 12:13:06.548467 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 174741us
I20260709 12:13:06.402983 23782 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:06.596673 23927 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.566382 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:06.417847 23930 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:06.549672 24594 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.567281 24808 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:06.605957 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.584126 24782 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:06.621409 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:06.610690 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.469273 24428 consensus_queue.cc:1243] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:06.514322 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.601743 24713 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:13:06.612630 23777 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.601758 24593 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:06.615932 24643 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.616721 24789 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.509562 24543 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.617249 24765 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.631494 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.632304 23777 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:06.633445 24765 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.618162 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:06.634433 23854 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:06.635919 23854 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:06.624145 23781 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:13:06.628194 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 154866us
I20260709 12:13:06.636206 23976 raft_consensus.cc:359] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.637537 23852 raft_consensus.cc:2368] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
W20260709 12:13:06.638077 23899 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.581s (SENT)
I20260709 12:13:06.638504 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.639994 24593 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:06.641439 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.643749 23976 raft_consensus.cc:385] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:13:06.644178 23976 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Initialized, Role: FOLLOWER
I20260709 12:13:06.644866 23976 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.643846 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.647152 23976 ts_tablet_manager.cc:1434] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Time spent starting tablet: real 0.061s	user 0.004s	sys 0.000s
I20260709 12:13:06.647656 23781 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
W20260709 12:13:06.648296 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.648892 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.650199 23781 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.652772 23854 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:06.618795 24704 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.613854 23928 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.633392 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.662303 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.665860 23781 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.622175 24594 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.558029 24758 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 12:13:06.659690 23776 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.623486 24808 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.669269 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.663237 24783 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.864913 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.869334 23785 raft_consensus.cc:2376] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2. Re-sending same reply.
I20260709 12:13:06.870826 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:06.872515 23899 leader_election.cc:336] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.726s (SENT)
I20260709 12:13:06.872227 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.679888 23777 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.879988 23899 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.881807 23784 raft_consensus.cc:2368] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3.
I20260709 12:13:06.739841 24631 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:06.750628 24515 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.829990 24580 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.889950 24580 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.892395 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.893107 24580 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.679728 24632 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:06.884402 24631 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:06.894388 24632 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.894645 24631 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.885627 24713 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:06.865499 24789 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.893247 23777 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.881824 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.886427 24594 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:06.885955 23899 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.949s (SENT)
I20260709 12:13:06.878731 23782 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.897361 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.897393 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.900481 23785 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:13:06.882565 24657 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.900113 24631 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:06.901695 24657 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.903505 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.903832 24765 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:06.904109 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.904485 23777 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:06.904661 24765 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:06.904932 23785 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.905036 24765 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:06.905520 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.905831 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.906457 23777 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.906041 24765 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:06.906579 23785 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:06.907119 24657 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.907375 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.907835 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.907209 24765 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.825529 24630 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.908805 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.915710 24808 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.916803 24783 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.917382 23928 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:06.897106 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.900861 23776 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:13:06.913216 24713 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:06.912086 24643 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.920542 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.929768 23783 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:06.928902 24704 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:13:06.931643 23778 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.932309 24704 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:06.932641 24704 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:06.930871 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44947.9us
W20260709 12:13:06.935649 23899 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.737s (SENT)
I20260709 12:13:06.936120 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.932996 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.926429 24783 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:13:06.958040 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.944790 24713 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:06.954109 23776 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.961830 24657 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:06.952605 24704 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.963075 24657 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:06.963505 24704 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:06.963900 24657 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:06.965139 24657 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:06.965723 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.967216 23783 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:06.970431 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:06.979152 23899 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.098s (SENT)
I20260709 12:13:06.980181 24630 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:06.980237 23899 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.981222 24726 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:06.982146 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:06.982890 23781 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:06.983348 23899 leader_election.cc:336] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.148s (SENT)
I20260709 12:13:06.983793 23899 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:06.984150 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:06.984833 23779 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:06.987676 24726 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:06.979046 23778 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:07.006788 23779 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:07.012460 24716 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.013132 24716 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.015157 24716 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.018813 23859 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:07.021085 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.021862 23857 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:07.023427 24783 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.024140 24783 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.027190 24783 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.029227 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.029883 23780 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.034307 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:07.035101 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.036556 23931 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.038522 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:07.051944 23751 leader_election.cc:400] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:07.052358 23751 leader_election.cc:403] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:13:07.053754 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.878s (SENT)
I20260709 12:13:07.054313 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:07.055550 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.056164 23929 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.057998 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.058964 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.062801 23931 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:07.064687 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.065330 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 97) took 2212 ms. Trace:
I20260709 12:13:07.067018 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.035065 23925 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:07.099462 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.100665 23779 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.047463 24808 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.114275 23929 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:13:07.105831 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62695.2us
I20260709 12:13:07.071967 24759 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:07.078949 24631 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:07.081269 24783 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.100143 24817 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:07.123274 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.072243 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.100457 24789 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:07.088151 23683 rpcz_store.cc:276] 0709 12:13:04.852612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.852789 (+   177us) service_pool.cc:224] Handling call
0709 12:13:04.958948 (+106159us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:07.065207 (+2106259us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:05.635211 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:05.732965 (+ 97754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:05.750518 (+ 17553us) write_op.cc:435] Acquired schema lock
0709 12:13:05.752918 (+  2400us) tablet.cc:662] Decoding operations
0709 12:13:05.798887 (+ 45969us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:05.799005 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:05.799038 (+    33us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:13:05.799706 (+   668us) tablet.cc:701] Row locks acquired
0709 12:13:05.799741 (+    35us) write_op.cc:260] PREPARE: finished
0709 12:13:05.799929 (+   188us) write_op.cc:270] Start()
0709 12:13:05.800080 (+   151us) write_op.cc:276] Timestamp: P: 1783599185799847 usec, L: 0
0709 12:13:05.800101 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:13:05.838866 (+ 38765us) log.cc:844] Serialized 2848 byte log entry
0709 12:13:06.289467 (+450601us) op_driver.cc:464] REPLICATION: finished
0709 12:13:06.513943 (+224476us) write_op.cc:301] APPLY: starting
0709 12:13:06.514037 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:06.514240 (+   203us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:06.549149 (+ 34909us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:06.667270 (+118121us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:06.667455 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:13:06.667496 (+    41us) write_op.cc:312] APPLY: finished
0709 12:13:06.667782 (+   286us) log.cc:844] Serialized 58 byte log entry
0709 12:13:06.667944 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:06.668052 (+   108us) write_op.cc:454] Released schema lock
0709 12:13:06.668092 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:06.668424 (+   332us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:06.668464 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39862,"reported_tablets":24,"spinlock_wait_cycles":41960704,"child_traces":[["op",{"apply.queue_time_us":179443,"apply.run_cpu_time_us":3836,"apply.run_wall_time_us":154514,"num_ops":7,"prepare.queue_time_us":158737,"prepare.run_cpu_time_us":5459,"prepare.run_wall_time_us":204139,"replication_time_us":489227,"thread_start_us":75860,"threads_started":2}]]}
I20260709 12:13:07.116289 23778 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.081908 24758 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:13:07.118335 24818 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:07.137382 24682 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:07.106595 24594 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:13:07.120117 24726 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:13:07.126899 24590 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 12:13:07.151285 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.892s (SENT)
W20260709 12:13:07.152277 23899 leader_election.cc:336] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.902s (SENT)
I20260709 12:13:07.142795 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.167809 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.168757 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.117947 24808 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:07.179960 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55569us
I20260709 12:13:07.135229 24789 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.141409 23680 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:07.198249 23680 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:07.134635 23776 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.143818 24428 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:07.139043 24759 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:07.146548 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:07.139055 24631 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.147786 24818 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.148394 24817 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:07.316843 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 164596us
W20260709 12:13:07.302557 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:07.235487 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:07.237411 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:07.321434 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 112893us
I20260709 12:13:07.321825 23782 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.322664 24817 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.322973 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:07.292681 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.222373 23680 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:07.320828 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.327766 23860 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:07.329777 23860 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:07.195551 24643 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:07.343118 23680 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:07.345005 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:07.336287 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:07.197549 24543 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:07.198414 24515 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:07.199412 24789 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.348052 23680 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.323792 24713 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:07.206626 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.349370 24767 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:07.350165 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.350731 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.349454 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.356552 24589 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:13:07.364423 23778 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.364850 24589 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:07.365181 24589 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:07.365433 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.365844 24789 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.220489 23780 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.366328 23778 raft_consensus.cc:2410] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:07.366029 24589 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.357188 24775 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:13:07.367691 23778 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:13:07.356161 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.358469 23776 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.336817 23928 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:07.360741 24767 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:07.370020 24767 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:07.359666 24829 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.066s	user 0.000s	sys 0.003s
I20260709 12:13:07.370333 24631 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.370822 24767 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.372030 24767 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.367184 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.370217 23776 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:07.336613 24759 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.374010 23776 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:07.375720 24643 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.375854 23778 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.354408 23680 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:07.376554 24515 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.341885 24817 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:07.363041 24827 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:13:07.378324 24515 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.378809 24817 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:07.358198 24783 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:07.376737 24643 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.377002 24543 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.278280 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.341650 24428 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:07.395469 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.341080 23782 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 0.
I20260709 12:13:07.402486 23785 raft_consensus.cc:2410] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:07.367158 24589 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.404623 23930 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:07.357800 24818 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.405817 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:07.467895 24759 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:07.383864 24817 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:07.471431 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 148341us
I20260709 12:13:07.475973 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.477169 24682 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:07.477609 24682 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.477952 24682 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.480916 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:07.437633 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.401499 23783 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.492316 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 338) took 1390 ms. Trace:
I20260709 12:13:07.501576 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.502137 24759 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.504007 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:07.472236 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.521553 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.524505 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.532085 23929 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:07.511497 23784 rpcz_store.cc:276] 0709 12:13:06.101802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.971338 (+869536us) service_pool.cc:224] Handling call
0709 12:13:06.971666 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.972020 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.972048 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.972490 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:13:07.075852 (+103362us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:07.133147 (+ 57295us) log.cc:844] Serialized 14 byte log entry
0709 12:13:07.135295 (+  2148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.135368 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.362466 (+227098us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:07.487176 (+124710us) raft_consensus.cc:1685] finished
0709 12:13:07.492050 (+  4874us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.492227 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25890,"prepare.queue_time_us":87441,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":7574,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":202,"spinlock_wait_cycles":14149632,"thread_start_us":102994,"threads_started":1,"wal-append.queue_time_us":224793}
I20260709 12:13:07.527079 23928 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:07.499441 23778 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:13:07.521586 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.540541 23860 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.543812 23860 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.405951 24428 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.529745 24716 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:07.528023 23776 raft_consensus.cc:2376] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4. Re-sending same reply.
I20260709 12:13:07.545068 24639 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:07.519654 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.568827 23928 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.577944 23929 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:07.556012 24716 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.525470 24759 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.585207 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.585824 23784 raft_consensus.cc:2376] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:07.513921 24543 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.590404 24842 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:13:07.595865 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.596452 23784 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:07.583827 24716 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.585083 24639 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.585487 23782 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:07.604599 24759 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:07.613294 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.614323 23784 raft_consensus.cc:2410] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:13:07.621408 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.106s (SENT)
I20260709 12:13:07.593441 24726 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:13:07.594641 24723 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:13:07.601842 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:07.634117 23899 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.198s (SENT)
I20260709 12:13:07.636608 23899 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:07.613544 24794 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:13:07.644009 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.644631 23784 raft_consensus.cc:2376] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:07.645498 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.649905 23776 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.650939 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.651662 23776 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:07.652611 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.657632 23933 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:13:07.664196 24844 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:13:07.667313 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 87238.6us
W20260709 12:13:07.668428 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.699s (SENT)
I20260709 12:13:07.661667 24759 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:07.651271 24817 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:07.674531 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.060s (SENT)
I20260709 12:13:07.677377 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:07.671218 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.686002 23776 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.676534 24682 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.690650 24682 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:07.696146 24726 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:07.677183 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.714196 24817 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:07.714522 24848 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 12:13:07.714849 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.715395 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.717664 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.719026 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:07.711992 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:07.728953 24833 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.435s	user 0.005s	sys 0.000s
I20260709 12:13:07.730116 23784 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:07.708057 24639 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.743290 24808 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.745030 23780 raft_consensus.cc:2410] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:13:07.747370 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.726330 23782 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:13:07.754042 23931 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:13:07.754227 23825 leader_election.cc:336] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.092s (SENT)
I20260709 12:13:07.756419 23825 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:07.759232 24808 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:07.801159 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.804900 23780 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.808878 24789 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.809722 24716 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.811489 23854 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:07.846316 23858 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.846764 24716 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:07.848477 23783 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.864598 23851 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:07.875031 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:07.876382 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.875918 23783 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:07.878492 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.878881 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 371) took 1379 ms. Trace:
I20260709 12:13:07.879386 23783 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.880843 23781 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:13:07.881266 23781 pending_rounds.cc:85] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 2
I20260709 12:13:07.881470 23781 pending_rounds.cc:107] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:13:07.881712 23781 raft_consensus.cc:2889] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:07.882302 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.882438 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.882915 23782 raft_consensus.cc:2376] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:07.883217 23777 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.881548 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.886443 24837 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:07.886878 23780 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.888641 23859 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.879280 23785 rpcz_store.cc:276] 0709 12:13:06.499935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.907469 (+407534us) service_pool.cc:224] Handling call
0709 12:13:07.878654 (+971185us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540041c154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:07.878820 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135487360}
I20260709 12:13:07.891894 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 107) took 1547 ms. Trace:
I20260709 12:13:07.892269 23680 rpcz_store.cc:276] 0709 12:13:06.364924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.383670 (+ 18746us) service_pool.cc:224] Handling call
0709 12:13:07.891809 (+1508139us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:07.605277 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:07.605852 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:07.605894 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:13:07.605911 (+    17us) tablet.cc:662] Decoding operations
0709 12:13:07.606740 (+   829us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:07.606798 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:07.606826 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:13:07.607306 (+   480us) tablet.cc:701] Row locks acquired
0709 12:13:07.607326 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:13:07.607462 (+   136us) write_op.cc:270] Start()
0709 12:13:07.607566 (+   104us) write_op.cc:276] Timestamp: P: 1783599187607418 usec, L: 0
0709 12:13:07.608775 (+  1209us) op_driver.cc:348] REPLICATION: starting
0709 12:13:07.645372 (+ 36597us) log.cc:844] Serialized 2459 byte log entry
0709 12:13:07.697257 (+ 51885us) op_driver.cc:464] REPLICATION: finished
0709 12:13:07.780435 (+ 83178us) write_op.cc:301] APPLY: starting
0709 12:13:07.780521 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:07.780723 (+   202us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:07.780846 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:07.781591 (+   745us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:07.781814 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:13:07.781864 (+    50us) write_op.cc:312] APPLY: finished
0709 12:13:07.782110 (+   246us) log.cc:844] Serialized 52 byte log entry
0709 12:13:07.782298 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:07.782411 (+   113us) write_op.cc:454] Released schema lock
0709 12:13:07.782462 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:07.782816 (+   354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:07.782849 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28513,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":82919,"apply.run_cpu_time_us":2372,"apply.run_wall_time_us":2372,"num_ops":6,"prepare.queue_time_us":101367,"prepare.run_cpu_time_us":3536,"prepare.run_wall_time_us":40396,"replication_time_us":83025,"thread_start_us":105027,"threads_started":2}]]}
W20260709 12:13:07.899204 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98123.2us
I20260709 12:13:07.901010 24643 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:07.902168 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.903836 23780 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:07.905140 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.906239 23782 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.906420 23777 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.915678 23784 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.908720 24837 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.918002 24809 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:07.918232 24837 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:07.919822 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:07.920388 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.920961 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.921638 23785 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.921937 24837 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:07.922411 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.924125 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.924952 23778 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.925091 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:07.925741 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.926752 24716 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:07.927273 24716 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:07.935221 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.930411 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:07.929226 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:07.931063 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.929924 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.941416 23779 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:07.913130 23681 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 2 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.916950 23781 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:13:07.917647 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.944553 23781 pending_rounds.cc:85] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 1
I20260709 12:13:07.944857 23781 pending_rounds.cc:107] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:13:07.945113 23781 raft_consensus.cc:2889] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:07.923506 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.946774 23785 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:07.936522 23784 raft_consensus.cc:2376] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:07.954054 23681 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.956214 23681 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.959362 23681 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.961203 23681 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.973639 23931 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:07.967623 23681 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:07.972790 23825 leader_election.cc:336] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.537s (SENT)
I20260709 12:13:07.975047 23681 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.977185 23681 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:07.978637 23681 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:07.980335 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.981069 23781 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:07.926878 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.983845 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.983978 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.932286 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:07.931838 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.988058 23781 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 3. Ops: []
I20260709 12:13:07.928149 24643 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:07.989001 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:07.989737 23779 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:07.991391 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.948917 24766 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:07.920408 23776 raft_consensus.cc:2376] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1. Re-sending same reply.
I20260709 12:13:07.955837 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:07.972304 24704 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.999055 23777 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:07.950119 23783 raft_consensus.cc:2376] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
W20260709 12:13:08.004868 24872 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:13:08.024488 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:07.982628 24726 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:13:08.054818 24643 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.058269 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.058560 23777 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:08.063670 23779 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.065665 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.066886 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 91381.7us
I20260709 12:13:08.052204 23930 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.001063 24765 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:13:08.004386 24704 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.066172 24716 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.076970 24704 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:08.078694 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.058421 24825 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:08.001595 24766 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.084474 23784 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:13:08.071671 24867 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:13:08.108342 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.070431 24639 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:13:08.121318 23781 raft_consensus.cc:1217] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:13:08.121932 23781 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:13:08.122848 24765 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:13:08.114370 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.123260 24765 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:08.127471 24765 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.128647 24765 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:08.129892 24766 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:08.134225 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 155447us
I20260709 12:13:08.136322 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.124352 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.141057 23780 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.146507 23783 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:08.148591 23779 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:08.156540 23929 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:13:08.182106 23825 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.559s (SENT)
I20260709 12:13:08.183480 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:08.182106 23899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.581s (SENT) [suppressed 85 similar messages]
I20260709 12:13:08.185730 24643 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.189656 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.190157 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.190310 23780 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:08.191952 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.192760 23781 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.198799 24639 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.201756 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:08.206284 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.564s (SENT)
I20260709 12:13:08.206491 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:08.210094 23929 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:08.212865 23927 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:08.223179 23783 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:13:08.243788 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.571s (SENT)
W20260709 12:13:08.257773 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72070.8us
I20260709 12:13:08.203437 23780 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1.
W20260709 12:13:08.262769 23899 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.608s (SENT)
I20260709 12:13:08.266008 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:08.266662 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.269928 24880 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:13:08.290235 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.292277 23680 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:08.338599 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.284286 24842 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:08.328644 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:08.391086 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.406553 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.407316 23784 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:13:08.407377 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 144664us
I20260709 12:13:08.408078 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.408254 23899 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.835s (SENT)
I20260709 12:13:08.408752 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:08.408778 23785 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:13:08.409408 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 165675us
I20260709 12:13:08.411322 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.412173 23785 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.413812 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.414528 23785 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.415522 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.415800 24716 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.416394 24716 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.418973 24716 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:08.416409 23783 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.420321 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.420552 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.421048 23784 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.421128 23785 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.424422 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:08.425096 23929 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:08.434460 24817 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:08.435288 24852 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:08.437812 23824 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:08.439373 24809 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:08.439949 24809 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.440340 24809 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.447562 24809 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.449741 24809 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:08.452667 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:08.453289 23933 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:08.456470 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.457345 23781 raft_consensus.cc:2410] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:08.457247 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 92) took 3136 ms. Trace:
W20260709 12:13:08.457420 23825 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.535s (SENT)
I20260709 12:13:08.459249 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.460834 23681 rpcz_store.cc:276] 0709 12:13:05.321246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.321387 (+   141us) service_pool.cc:224] Handling call
0709 12:13:08.457187 (+3135800us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:08.074751 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:08.075322 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:08.075362 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:13:08.075380 (+    18us) tablet.cc:662] Decoding operations
0709 12:13:08.076404 (+  1024us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:08.076475 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:08.076506 (+    31us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:13:08.077280 (+   774us) tablet.cc:701] Row locks acquired
0709 12:13:08.077301 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:13:08.077480 (+   179us) write_op.cc:270] Start()
0709 12:13:08.077625 (+   145us) write_op.cc:276] Timestamp: P: 1783599188077420 usec, L: 0
0709 12:13:08.077648 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:13:08.078462 (+   814us) log.cc:844] Serialized 3618 byte log entry
0709 12:13:08.114028 (+ 35566us) op_driver.cc:464] REPLICATION: finished
0709 12:13:08.123749 (+  9721us) write_op.cc:301] APPLY: starting
0709 12:13:08.130912 (+  7163us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:08.131155 (+   243us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:08.133457 (+  2302us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:08.136533 (+  3076us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:08.136739 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:13:08.136783 (+    44us) write_op.cc:312] APPLY: finished
0709 12:13:08.137049 (+   266us) log.cc:844] Serialized 70 byte log entry
0709 12:13:08.138182 (+  1133us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.138311 (+   129us) write_op.cc:454] Released schema lock
0709 12:13:08.138364 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:08.138725 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.138749 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3352,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":1007,"apply.run_cpu_time_us":3628,"apply.run_wall_time_us":14976,"num_ops":9,"prepare.queue_time_us":100,"prepare.run_cpu_time_us":4059,"prepare.run_wall_time_us":6781,"replication_time_us":18136}]]}
I20260709 12:13:08.465201 23928 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:08.466073 23926 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:08.467983 24884 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:08.468052 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59808.6us
W20260709 12:13:08.469347 23899 leader_election.cc:336] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.686s (SENT)
I20260709 12:13:08.469746 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:08.470754 23899 leader_election.cc:336] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.704s (SENT)
I20260709 12:13:08.471119 23899 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:08.471467 24594 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.472095 24715 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.467417 24852 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:08.473388 23899 leader_election.cc:336] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.705s (SENT)
W20260709 12:13:08.485078 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.486126 23933 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
W20260709 12:13:08.506922 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:08.507124 23933 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:08.508426 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.461728 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.510051 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52648.4us
I20260709 12:13:08.509227 23781 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.512259 23824 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:08.517138 24807 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:13:08.523775 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.524649 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.528093 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.530085 23780 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.520493 24789 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:13:08.520187 24857 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:08.512814 24643 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:13:08.526077 23926 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:08.528792 23781 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.542467 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.543156 23781 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:08.545783 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.546622 23781 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.548503 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.548748 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.549176 23780 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.549538 23785 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.549929 23779 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:13:08.550122 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.550340 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.550707 23783 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.550947 23784 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
W20260709 12:13:08.547111 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.558326 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.559916 23776 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.568959 23781 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:08.570859 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:08.573444 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.605s (SENT)
I20260709 12:13:08.573928 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.574991 24817 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:08.576575 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.682s (SENT)
I20260709 12:13:08.576658 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.577313 23781 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
W20260709 12:13:08.578989 23825 leader_election.cc:336] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.659s (SENT)
W20260709 12:13:08.579229 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54951.4us
I20260709 12:13:08.579806 23825 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.579259 24857 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:08.581161 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.581522 24857 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.583038 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.584084 23784 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:13:08.588060 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.591008 23783 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:08.595160 23683 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.604383 23683 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.606525 23683 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.608018 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:08.618371 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.699s (SENT)
I20260709 12:13:08.623898 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.625357 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.625999 23783 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.625682 23683 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } }
I20260709 12:13:08.628768 23683 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.634629 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.634866 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.635550 23780 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.636117 23779 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:08.636391 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.637148 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.638188 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.617601 24817 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.642984 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.639488 24808 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:13:08.641052 23785 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:08.643658 23784 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:08.645690 23783 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:08.656913 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.666249 23777 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.666646 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.667217 23783 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:08.635563 23782 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.663003 23683 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
W20260709 12:13:08.672067 23825 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.489s (SENT)
I20260709 12:13:08.672665 23825 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.673794 24643 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:08.669349 23683 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.676440 23683 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.678501 23683 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.679811 23927 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:08.679648 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.682852 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.687748 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.688086 23785 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.692638 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:08.694842 23899 leader_election.cc:336] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.819s (SENT)
I20260709 12:13:08.698794 23783 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.699780 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.699945 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.700775 23782 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:13:08.701399 23779 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.701038 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.702111 23783 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 3 because replica has last-logged OpId of term: 2 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:13:08.702436 23683 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.705039 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.705236 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.705188 23683 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.706315 23782 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.707309 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.705801 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.707479 23683 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.708451 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.709005 23785 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.709815 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.710408 23785 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.710145 24852 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:08.711125 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.711861 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.712443 23779 raft_consensus.cc:2376] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3. Re-sending same reply.
I20260709 12:13:08.713269 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.713901 23779 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:08.714214 23785 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
W20260709 12:13:08.717867 23899 leader_election.cc:336] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.702s (SENT)
W20260709 12:13:08.704174 23825 leader_election.cc:336] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.839s (SENT)
I20260709 12:13:08.719406 23683 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.722249 23777 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.724824 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.725445 23777 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:13:08.728767 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.864s (SENT)
I20260709 12:13:08.729295 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.730970 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.834916 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.840492 23780 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.786233 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.848691 23779 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:08.849750 23783 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:08.831099 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.756279 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.776361 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.905s (SENT)
I20260709 12:13:08.785117 23784 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.844877 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.852451 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:08.852466 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 76143.2us
W20260709 12:13:08.853648 23899 leader_election.cc:336] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.627s (SENT)
I20260709 12:13:08.811750 23782 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.856798 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.858006 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.858722 23782 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.859445 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.853174 23784 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:08.843981 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 115267us
I20260709 12:13:08.865152 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.851261 23785 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.866767 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.867518 23777 raft_consensus.cc:2410] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:08.868367 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.868685 23781 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:08.868871 23777 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:08.846976 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.869701 23781 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:08.865274 23825 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.884s (SENT)
I20260709 12:13:08.865911 23784 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
W20260709 12:13:08.872365 23825 leader_election.cc:336] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.884s (SENT)
I20260709 12:13:08.758267 23683 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.872781 23825 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:08.873775 23825 leader_election.cc:336] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.905s (SENT)
I20260709 12:13:08.874150 23825 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:08.874460 23683 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.875456 23683 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.876791 23683 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.771811 24716 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:08.878190 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.914s (SENT)
I20260709 12:13:08.880769 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.881146 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:08.877903 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.881767 23785 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:08.880808 23825 leader_election.cc:336] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.929s (SENT)
I20260709 12:13:08.878679 23683 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.886349 23683 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:08.884411 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.884698 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.880199 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.888808 23777 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.893621 23783 raft_consensus.cc:2376] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
W20260709 12:13:08.889638 24906 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 12:13:08.873813 24643 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.889817 24713 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:08.877894 24631 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:08.891140 23899 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.918s (SENT)
I20260709 12:13:08.896090 23899 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:08.892565 23683 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.893986 23779 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:08.899678 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.900492 23779 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.900765 24837 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:08.898116 23683 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:08.902789 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.903280 23683 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.905112 23683 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.906388 24590 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:08.906906 24590 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:08.910555 24590 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:08.910784 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.912138 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.912786 23779 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.908568 23683 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.919174 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:08.919898 23858 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 1.
I20260709 12:13:08.924367 23900 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:08.926167 24842 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:08.926594 24842 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:08.926952 24842 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:08.929653 23683 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.932210 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:08.930058 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.933717 23785 raft_consensus.cc:2376] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:08.937024 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.937732 23779 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:13:08.941238 24903 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.128s	user 0.002s	sys 0.000s
W20260709 12:13:08.946026 23899 leader_election.cc:336] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.930s (SENT)
I20260709 12:13:08.960657 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:08.968883 23899 leader_election.cc:336] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.006s (SENT)
I20260709 12:13:08.968787 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.970398 23779 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.975978 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.976733 23785 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:08.977648 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.978463 23779 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:08.972640 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.979974 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.980712 23783 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 1: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:08.986775 24905 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.131s	user 0.000s	sys 0.002s
I20260709 12:13:08.988883 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:08.989264 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.989670 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:08.984400 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.994495 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:08.983685 23778 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:08.997403 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:08.998103 23779 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:13:09.003647 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.004693 23779 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:09.006683 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.015971 23931 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:09.016394 24842 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:09.020452 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.020996 24910 log.cc:927] Time spent T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:13:09.022807 24842 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:09.020431 23782 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:09.024377 24428 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.025023 24428 consensus_queue.cc:1243] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:09.026155 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:09.026755 23851 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.028008 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.028599 23778 raft_consensus.cc:2376] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:09.029049 23783 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:09.029443 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.030105 23782 raft_consensus.cc:2376] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:09.033196 23776 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:09.034473 23929 pending_rounds.cc:85] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 1
I20260709 12:13:09.034750 23929 pending_rounds.cc:107] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:13:09.034952 23929 raft_consensus.cc:2889] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.044366 23779 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:09.052024 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.052750 23776 raft_consensus.cc:2376] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3. Re-sending same reply.
I20260709 12:13:09.053031 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.053731 23779 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:09.058000 23899 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.023s (SENT)
W20260709 12:13:09.059062 23899 leader_election.cc:336] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.040s (SENT)
I20260709 12:13:09.060842 23779 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:09.053821 23851 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:09.063104 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.063474 23776 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:13:09.063916 23776 pending_rounds.cc:85] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 1
I20260709 12:13:09.064261 23776 pending_rounds.cc:107] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:13:09.064534 23776 raft_consensus.cc:2889] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.063900 23900 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:09.065992 24855 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:09.066489 24855 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:09.067387 24855 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.068369 23779 raft_consensus.cc:2376] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3. Re-sending same reply.
W20260709 12:13:09.070597 23825 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.055s (SENT)
I20260709 12:13:09.071074 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:09.071508 23825 leader_election.cc:336] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.066s (SENT)
I20260709 12:13:09.072160 23825 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.073158 24630 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.071045 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.074440 23782 raft_consensus.cc:2376] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 1. Re-sending same reply.
W20260709 12:13:09.075498 23825 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.098s (SENT)
I20260709 12:13:09.076010 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.077169 24630 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.078780 23776 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:09.091984 23783 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:09.092099 23782 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:09.093289 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.093961 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.094130 23776 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:09.095875 23782 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:09.097402 23782 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:09.099607 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.100126 23779 raft_consensus.cc:2376] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3. Re-sending same reply.
I20260709 12:13:09.101315 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.101467 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.102228 23779 raft_consensus.cc:2376] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:09.108425 23779 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.108916 23779 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:09.109823 23779 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.110893 23779 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.113474 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.113749 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.114316 23776 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:09.114416 23782 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:09.129495 23779 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:13:09.137936 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.139935 23782 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:09.139107 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.141297 23779 pending_rounds.cc:85] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 2
I20260709 12:13:09.141644 23779 pending_rounds.cc:107] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:13:09.141772 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.160596 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.161007 23783 raft_consensus.cc:2376] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:09.143299 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:09.145676 23899 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.130s (SENT)
I20260709 12:13:09.161927 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.152595 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.162307 23783 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:09.153093 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.162616 23782 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:09.164216 24855 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.169718 23778 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:09.175701 23783 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:13:09.178237 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.162390 23779 raft_consensus.cc:2889] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.181510 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.183609 23926 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:09.188050 23783 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:09.197367 23899 leader_election.cc:336] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.080s (SENT)
I20260709 12:13:09.199167 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.205994 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.213991 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.216382 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.221048 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.223978 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.226413 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.228609 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.230804 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:09.232694 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.233203 23929 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:09.234953 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.235625 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:09.236650 23825 leader_election.cc:336] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.373s (SENT)
I20260709 12:13:09.237097 23825 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.238132 24428 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.242161 24855 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.244298 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.245419 24594 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:09.247807 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.245476 24842 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:13:09.248834 24855 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:09.249208 24855 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:09.249655 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.249974 24855 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.251331 24855 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:09.251741 23825 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.111s (SENT)
W20260709 12:13:09.258808 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.534s (SENT)
W20260709 12:13:09.264012 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.541s (SENT)
I20260709 12:13:09.266394 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:09.268034 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.543s (SENT)
I20260709 12:13:09.271170 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.274029 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.277143 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:09.285279 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.946s (SENT)
I20260709 12:13:09.285889 23681 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:09.303939 23929 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:09.311159 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:09.319806 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.321013 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.325500 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.326656 24855 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.329802 23899 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.330994 24855 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.333503 24855 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.334546 24842 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.334928 24842 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:09.336110 24842 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.337486 24842 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.337641 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:09.341125 23899 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.702s (SENT)
W20260709 12:13:09.345082 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.351253 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.356205 24842 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.359239 24855 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.360652 24855 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.364526 24713 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.366600 24713 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.369985 24713 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.378340 24713 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.380569 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.383116 24630 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.384258 24630 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:09.384589 24630 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.385447 24630 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.386751 24630 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.389477 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.395946 24852 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.397374 24852 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.398662 24852 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.407778 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:09.402084 23683 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:09.381791 24713 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.414449 24639 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:13:09.414991 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.422041 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.434296 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.439239 23825 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.440131 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.429649 24789 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:13:09.423234 24852 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:13:09.426380 24885 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:13:09.440969 24515 raft_consensus.cc:2764] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:09.435819 24716 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:13:09.426244 24884 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:13:09.442678 24807 raft_consensus.cc:2764] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:09.427994 24837 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:13:09.427870 24631 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:13:09.429345 24428 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:13:09.443858 24765 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.477228 24765 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.479383 24713 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:09.484063 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43965.9us
I20260709 12:13:09.485339 23825 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.491326 24808 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:09.479461 24765 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.491812 24808 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:09.492204 24808 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:09.493705 23749 connection.cc:256] RPC call timeout handler was delayed by -2.6083240910002132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 12:13:09.494246 23749 connection.cc:256] RPC call timeout handler was delayed by -2.5894240910001827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 12:13:09.494540 23749 connection.cc:256] RPC call timeout handler was delayed by -2.5750240910001594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 12:13:09.494807 23749 connection.cc:256] RPC call timeout handler was delayed by -2.5597240910001346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
I20260709 12:13:09.481657 23681 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:09.495118 23749 connection.cc:256] RPC call timeout handler was delayed by -2.5399240910001026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 12:13:09.496395 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.496196 24857 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:09.534727 24765 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.539812 24765 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:09.495467 23749 connection.cc:256] RPC call timeout handler was delayed by -2.4976240910000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 12:13:09.542251 23749 connection.cc:256] RPC call timeout handler was delayed by -2.3815240910000739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
W20260709 12:13:09.542572 23749 connection.cc:256] RPC call timeout handler was delayed by -2.3140240910001921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:13:09.542840 23749 connection.cc:256] RPC call timeout handler was delayed by -2.2825240910001412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 12:13:09.543093 23749 connection.cc:256] RPC call timeout handler was delayed by -2.2636240910001106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:13:09.543361 23749 connection.cc:256] RPC call timeout handler was delayed by -2.2195240910000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 12:13:09.543634 23749 connection.cc:256] RPC call timeout handler was delayed by -2.1880240910002158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 12:13:09.535763 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 108993us
W20260709 12:13:09.547196 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 78736.2us
W20260709 12:13:09.543968 23749 connection.cc:256] RPC call timeout handler was delayed by -2.103424091000079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260709 12:13:09.553261 23749 connection.cc:256] RPC call timeout handler was delayed by -2.0620240910002394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 12:13:09.553604 23749 connection.cc:256] RPC call timeout handler was delayed by -2.0566240910002307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 12:13:09.553877 23749 connection.cc:256] RPC call timeout handler was delayed by -2.0557240910002292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 12:13:09.554158 23749 connection.cc:256] RPC call timeout handler was delayed by -2.0026240910001434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
W20260709 12:13:09.554436 23749 connection.cc:256] RPC call timeout handler was delayed by -1.8298240910000914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23730000000000004s for a response
I20260709 12:13:09.554689 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.558013 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:09.554723 23749 connection.cc:256] RPC call timeout handler was delayed by -1.6453240910002478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2578s for a response
W20260709 12:13:09.558845 23749 connection.cc:256] RPC call timeout handler was delayed by -1.6309240910002245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2594s for a response
W20260709 12:13:09.559168 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5220240910000484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2715s for a response
W20260709 12:13:09.559449 23749 connection.cc:256] RPC call timeout handler was delayed by -1.4752240910002001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2767s for a response
W20260709 12:13:09.559755 23749 connection.cc:256] RPC call timeout handler was delayed by -1.2322240910000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3037s for a response
I20260709 12:13:09.559958 24885 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.560367 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:09.561615 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.562529 24922 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.563815 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:09.568603 24855 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:09.569133 24855 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:09.569536 24855 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.573002 24921 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.558797 24713 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:13:09.574465 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:09.546077 24789 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:09.583245 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:09.601677 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 109746us
I20260709 12:13:09.580672 24837 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:09.604859 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.580228 24789 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.580992 24594 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:13:09.605100 24765 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.610572 24923 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:09.604362 24808 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:09.630517 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68966.4us
I20260709 12:13:09.631433 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.619349 24922 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.629187 24789 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.637465 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.639161 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.639515 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.646966 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.647529 24921 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.648597 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.636538 24765 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.654794 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.632550 24928 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.646821 24925 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.652840 24807 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:09.659960 24789 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.646065 24515 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:13:09.659444 24716 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:13:09.671469 24630 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:09.669636 24817 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:09.671757 24930 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.672701 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.664606 24931 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.671388 24808 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.674608 24765 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.673672 24630 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.684382 24428 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:13:09.675431 24807 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.689545 24428 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.689980 24807 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.690512 24428 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.696189 24630 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.697700 24630 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:09.699993 24809 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:09.700438 24809 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.675446 24857 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:13:09.701257 24809 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.673319 24927 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:09.678202 24643 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:13:09.722699 24809 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:09.737172 24855 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.738441 23926 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:09.761623 24855 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:09.771576 24817 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.772742 24817 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:09.777931 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 94586.1us
W20260709 12:13:09.786016 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 108704us
I20260709 12:13:09.793283 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:09.799163 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 126518us
I20260709 12:13:09.800509 24643 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:13:09.806687 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 158182us
I20260709 12:13:09.804600 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:09.807318 23749 leader_election.cc:343] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:13:09.807780 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.808574 23854 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:09.808897 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.809630 24925 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:09.805984 24543 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:13:09.810286 24925 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.812423 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.813498 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.815299 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.817243 23825 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.818960 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:09.821163 23731 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:13:09.819924 24925 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.813890 24808 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:13:09.817258 24428 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:13:09.820102 24817 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:09.824994 24817 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.825016 24789 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:09.825732 24789 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:09.826233 24789 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:09.817075 24643 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:13:09.825909 24817 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.824625 24631 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:09.827116 23825 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.829290 24631 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.830049 24925 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.815572 24931 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.830915 24808 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:09.830753 24631 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.831280 24808 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:09.831898 24631 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.832062 24808 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.833353 24808 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:09.834455 23825 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.835309 24807 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:09.837080 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:09.837476 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.835779 24937 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.837813 24765 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.835515 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.839178 24937 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:09.839519 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.839677 24937 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:09.840222 24937 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.841836 24543 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:09.841933 24932 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:09.842286 24543 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:09.842944 24543 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.844732 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.846338 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.846758 24934 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.847493 24934 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.853348 24934 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.854060 23749 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.854393 24934 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.854791 24934 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:09.854853 24931 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.856150 23749 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.856859 24765 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.857277 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.857973 24765 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.858402 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.859454 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.860272 24807 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.859323 24765 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.861800 24789 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.862267 24543 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.864253 24543 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.864388 24931 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.864645 24789 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.861261 24937 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.864936 24934 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.866036 23749 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.866273 24934 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.866984 24937 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:09.867168 24765 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.868427 23749 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.869141 24765 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.870275 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.870990 24765 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.871975 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.873099 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.873293 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.874502 23749 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.873769 24765 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.875202 24931 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.876653 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.877370 24765 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.877265 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:09.878919 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.879669 24765 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.880707 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.881495 24921 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.881927 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.882668 24931 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.883888 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.884641 24921 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:09.885704 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.887830 24921 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.888994 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.889806 24765 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.890062 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.890754 24921 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.892009 24921 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:09.892715 24921 consensus_queue.cc:1243] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:09.892453 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:09.893249 24925 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:09.893567 24925 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:09.894325 24925 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.896227 24925 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:09.904107 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.904954 24925 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.905390 23749 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.906069 24925 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.906469 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.907114 24925 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:09.907696 24921 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:13:09.908639 23749 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.909358 24921 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:09.909713 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:09.910369 24921 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:09.894089 23681 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:09.911530 23681 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:09.916266 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.916991 23783 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:09.922972 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.923660 23783 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:09.924962 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.931192 23783 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:09.932337 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.932997 23779 raft_consensus.cc:2376] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:09.933223 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.934090 23782 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 1: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:09.934100 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.934736 23683 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:09.935459 23778 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:09.937069 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.937604 23778 raft_consensus.cc:2376] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:09.939224 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.945971 23778 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:09.947151 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.949522 23782 raft_consensus.cc:2376] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:09.948293 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.950538 23779 raft_consensus.cc:2376] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4. Re-sending same reply.
I20260709 12:13:09.948863 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:09.951232 23783 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:09.952529 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.958079 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:09.958817 23782 raft_consensus.cc:2376] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3. Re-sending same reply.
I20260709 12:13:09.962646 23851 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:09.964684 24933 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.965585 24933 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:09.965924 24933 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:09.966759 24933 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:09.968137 24933 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:09.974260 23776 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:10.005281 23854 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:10.007642 24932 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:10.008741 24855 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:10.009130 24855 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.010023 24855 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.011199 24855 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.034463 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:10.034601 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:10.037691 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:10.042691 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:10.045533 24594 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:13:10.045949 24932 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:10.082120 23749 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.046412 24926 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:10.045270 24631 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:13:10.069845 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:10.094579 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.095932 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.098059 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.098850 23782 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:10.085598 24931 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:10.083835 24932 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:10.114670 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.090646 24926 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.120333 23784 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:10.140753 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.112717 24932 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.125229 24789 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:10.143307 24765 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:10.167302 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.168490 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.168797 24926 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.169629 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.170246 23777 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:10.173941 23854 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:10.176649 23825 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.178484 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.179567 24789 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:10.180162 24937 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:10.181898 23825 leader_election.cc:400] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3.
I20260709 12:13:10.183554 24932 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.163748 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.198463 23785 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.189495 23782 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:10.162701 24594 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:13:10.226495 23785 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:10.231662 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.233067 24817 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:10.233577 24817 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:10.234282 24932 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:10.234493 24817 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:10.235726 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.245177 23779 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:10.260205 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 96536.2us
I20260709 12:13:10.254199 23782 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:10.274083 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:10.274729 23777 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.279409 24594 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:13:10.281464 24594 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:10.281821 24594 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.283226 23777 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:10.282909 24594 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.291934 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:10.302119 24594 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:10.313369 24765 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:10.363893 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.368954 24968 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:13:10.357525 24765 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.378381 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:10.400259 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:10.401611 24939 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:10.402087 24939 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.418255 23928 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.435073 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:10.455277 23928 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:10.465315 23858 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:10.434266 24926 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:10.450682 24971 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:10.455096 24765 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:10.454541 23926 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:10.460907 24933 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:13:10.444396 24939 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.475188 24971 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.482908 24921 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.495756 24543 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:13:10.528592 23776 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:10.529167 24987 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.529670 24987 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.492914 24921 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:10.562443 24977 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:13:10.564824 24981 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:13:10.565903 23777 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:10.496760 24643 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:10.567974 24543 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:10.576160 24543 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.532696 24982 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:10.585259 24987 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:10.587845 24982 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.589978 24982 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:10.577967 23926 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:10.587363 23681 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.532766 24971 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.596503 24543 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:10.656105 24992 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:10.656746 24992 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.647724 24926 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.673382 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:10.703542 23781 raft_consensus.cc:2393] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:13:10.717149 24991 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:13:10.666826 24921 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:10.723672 23776 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:10.703655 24594 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:10.710103 24926 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.729238 24926 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:10.782764 23966 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.22.25.126:39113, user_credentials={real_user=slave}} blocked reactor thread for 51042us
I20260709 12:13:10.792362 23783 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:10.793840 23779 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:10.801762 23783 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:10.799626 24990 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:13:10.731913 24594 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.854279 23933 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:10.856542 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:10.853868 22629 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 12:13:10.803218 24789 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:13:10.862624 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:10.858511 24921 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:10.869123 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55934.6us
I20260709 12:13:10.843845 24817 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:13:10.878896 24852 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:13:10.886817 24885 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:10.897840 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:10.902489 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:10.867022 23926 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:10.905361 24885 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:10.907579 24992 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:10.938162 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 44802.6us
I20260709 12:13:10.934695 24994 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.872603 24921 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.955971 24994 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.973666 25005 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:10.974826 24937 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:10.975251 24937 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:10.976064 24937 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.977246 24937 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:10.981163 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 111305us
W20260709 12:13:10.982564 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 125617us
I20260709 12:13:10.967356 24994 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:10.983752 24543 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:13:10.949404 24789 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:13:10.985288 24885 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.986951 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:10.994239 23683 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:11.003296 23783 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:11.004747 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:10.998752 24927 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.999480 24933 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.016027 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.008538 24885 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.008262 24594 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.020219 24933 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:11.012465 24980 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.523s	user 0.006s	sys 0.006s
I20260709 12:13:11.022421 24933 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.008672 24971 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.036728 24971 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.039181 24971 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.007730 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.044116 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.018810 24927 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.008967 24934 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.035877 24789 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.047881 24789 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.050153 24789 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.050627 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.051280 23929 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:11.052075 24921 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.040410 24713 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.053151 24713 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.047152 24934 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:10.984630 25006 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:13:11.048220 24927 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.026571 24543 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.062609 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 57930.8us
W20260709 12:13:11.062642 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49272.2us
I20260709 12:13:11.064180 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:11.066630 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.067353 23784 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.068806 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.069527 23858 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:11.070078 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.071434 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.074091 23784 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:11.072798 24927 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.075536 24927 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.075973 24937 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.062090 23926 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:11.077260 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.079193 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.057534 24543 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.073160 24931 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:11.086143 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.089947 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 512) took 1024 ms. Trace:
I20260709 12:13:11.090417 23776 rpcz_store.cc:276] 0709 12:13:10.065095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.726163 (+661068us) service_pool.cc:224] Handling call
0709 12:13:10.726502 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.726820 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.726847 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.746493 (+ 19646us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.821073 (+ 74580us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:10.860079 (+ 39006us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.037737 (+177658us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b180036fe80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:13:11.078409 (+ 40672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.078478 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.078557 (+    79us) raft_consensus.cc:1685] finished
0709 12:13:11.089737 (+ 11180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.089877 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23449,"prepare.queue_time_us":40025,"raft.queue_time_us":45824,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":66641,"spinlock_wait_cycles":25945344,"thread_start_us":68941,"threads_started":2,"wal-append.queue_time_us":67387}
I20260709 12:13:11.092672 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.093442 23824 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:11.093868 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.094949 23776 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:11.096550 23784 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.096482 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.097251 23776 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.098246 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:11.076464 24937 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.098430 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.099696 24931 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:11.100109 23858 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:11.101086 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.091420 24992 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:11.103294 23776 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:11.105307 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.105959 23776 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:11.106921 24713 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.101928 24789 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:13:11.101259 24643 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.108103 24643 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.102615 25000 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.108464 24643 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.110212 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 510) took 1045 ms. Trace:
I20260709 12:13:11.110579 23781 rpcz_store.cc:276] 0709 12:13:10.064667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.726251 (+661584us) service_pool.cc:224] Handling call
0709 12:13:10.746221 (+ 19970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.746723 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.746751 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.786032 (+ 39281us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.969028 (+182996us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:10.975557 (+  6529us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.986202 (+ 10645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.986281 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.110016 (+123735us) raft_consensus.cc:1685] finished
0709 12:13:11.110035 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.110151 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15092,"prepare.queue_time_us":6085,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":10374,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":19614,"thread_start_us":124558,"threads_started":1,"wal-append.queue_time_us":209990}
I20260709 12:13:11.112318 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.112442 24927 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.100715 23778 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:11.114885 24713 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.115270 24713 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.115576 25000 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.115669 24713 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.116006 25000 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.116828 24543 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.115408 23851 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:11.122691 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.139647 24933 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.140271 24933 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.141112 24713 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.141577 25000 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.142493 24933 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.143105 24713 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.145897 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.147960 23852 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.146446 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.148923 23858 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.143445 25000 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.161576 24789 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:11.162189 24789 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.164235 24789 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.145536 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 513) took 1070 ms. Trace:
W20260709 12:13:11.167814 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68584.4us
I20260709 12:13:11.168061 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.172516 23783 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:11.127962 25002 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.173278 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:11.174525 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:11.180856 25002 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.170650 23780 rpcz_store.cc:276] 0709 12:13:10.075489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.785783 (+710294us) service_pool.cc:224] Handling call
0709 12:13:10.827862 (+ 42079us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:10.920986 (+ 93124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.921021 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:10.921982 (+   961us) log.cc:844] Serialized 52 byte log entry
0709 12:13:10.955892 (+ 33910us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.998712 (+ 42820us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.024351 (+ 25639us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.037048 (+ 12697us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:13:11.078241 (+ 41193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.081036 (+  2795us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.145351 (+ 64315us) raft_consensus.cc:1685] finished
0709 12:13:11.145372 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.145484 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90401,"prepare.queue_time_us":8011,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":942,"raft.queue_time_us":108191,"raft.run_cpu_time_us":1106,"raft.run_wall_time_us":122204,"spinlock_wait_cycles":15410048,"thread_start_us":85696,"threads_started":2,"wal-append.queue_time_us":144179}
I20260709 12:13:11.178129 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.187708 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.191201 23858 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:13:11.195329 25022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:13:11.188211 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.183911 24937 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.196060 23855 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.179709 24543 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.179546 24789 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:11.186971 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 93596.2us
I20260709 12:13:11.189679 25002 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.206377 23858 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:11.206507 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.208123 23934 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:11.198387 24543 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.198663 24789 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.210700 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.216300 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.217104 23931 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:11.219638 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.221414 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.222069 23932 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:11.222240 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.222795 23927 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:11.214104 24543 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.224404 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 495) took 1162 ms. Trace:
I20260709 12:13:11.224857 23782 rpcz_store.cc:276] 0709 12:13:10.061485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.285562 (+224077us) service_pool.cc:224] Handling call
0709 12:13:10.285887 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.309092 (+ 23205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.310357 (+  1265us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.344561 (+ 34204us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.420239 (+ 75678us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:10.453292 (+ 33053us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.196015 (+742723us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b18002e0de0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:13:11.217221 (+ 21206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.217306 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.218857 (+  1551us) raft_consensus.cc:1685] finished
0709 12:13:11.224164 (+  5307us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.224332 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13272,"prepare.queue_time_us":44667,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":35,"raft.queue_time_us":10883,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":5967,"spinlock_wait_cycles":3530496,"thread_start_us":84255,"threads_started":2,"wal-append.queue_time_us":143090}
I20260709 12:13:11.215451 24789 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.230901 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.227491 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.246417 23926 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:13:11.234670 25024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:13:11.237993 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 509) took 1173 ms. Trace:
I20260709 12:13:11.238524 24933 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:11.248122 23777 rpcz_store.cc:276] 0709 12:13:10.064474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.566716 (+502242us) service_pool.cc:224] Handling call
0709 12:13:10.567005 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.581810 (+ 14805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.581840 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.590779 (+  8939us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.804273 (+213494us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:10.851827 (+ 47554us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.856046 (+  4219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.911975 (+ 55929us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:11.080247 (+168272us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b18002c27e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5522 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:13:11.237711 (+157464us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b5400166554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:11.237755 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.237799 (+    44us) raft_consensus.cc:1685] finished
0709 12:13:11.237811 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.237936 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39849,"prepare.queue_time_us":55673,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1257,"raft.queue_time_us":33727,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":36091,"spinlock_wait_cycles":43867776,"thread_start_us":212909,"threads_started":1,"wal-append.queue_time_us":240361}
I20260709 12:13:11.240940 23900 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:11.242717 23778 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:11.252794 24885 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.253192 24885 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.253556 24885 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:11.254213 25020 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.178s	user 0.004s	sys 0.000s
I20260709 12:13:11.262627 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.263377 23785 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.245853 25032 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.265426 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:11.265285 23900 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:11.266500 23784 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:11.267827 23899 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:11.266587 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.269831 23777 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.270901 25011 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.271277 25011 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.271644 25011 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.272981 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.273777 23782 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:11.267205 23900 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.286437 23900 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.290256 23785 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.276235 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.294423 25002 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.300258 24988 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:11.270495 25032 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.302788 25002 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.305878 25002 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:11.303515 25030 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:13:11.279832 24643 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.310474 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:11.313154 23777 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:11.302652 24992 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.316798 23855 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.314558 23778 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:11.312546 24971 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:13:11.319182 23854 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:11.329272 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.321739 24932 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.338004 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.338644 23779 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:11.359963 24926 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.360394 24543 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.361680 23858 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.362653 24543 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.365813 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.370030 24995 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:11.336997 24932 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.371948 24932 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.375207 24789 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.376078 24643 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.331308 24992 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.377892 24789 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.378991 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.379724 23780 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:11.382519 23855 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.359427 24885 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.386006 25011 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.359202 24982 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.361721 25032 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:11.407838 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 130945us
I20260709 12:13:11.420224 23824 leader_election.cc:400] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:11.420632 23824 leader_election.cc:403] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:11.407573 25033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:13:11.404696 24987 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.435472 25038 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:11.441584 24934 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.441815 24992 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.441016 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.442775 25009 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.443197 25009 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.443531 25009 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.360459 24926 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.446870 23928 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:11.448472 23900 leader_election.cc:400] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.449596 25011 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.451224 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.451655 23900 leader_election.cc:403] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.455708 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:11.456523 24995 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.456871 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.459003 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.463867 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:11.420761 24982 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.453596 24939 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.460435 24713 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:11.472210 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.446640 24987 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.468804 25032 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:11.413293 24932 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.469009 25011 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.497859 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.500958 23851 raft_consensus.cc:2368] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.474082 24988 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:11.502945 23900 leader_election.cc:400] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.473980 24713 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.504382 23900 leader_election.cc:403] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.496519 24885 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.495558 24934 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.510422 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.504624 22629 server_base.cc:1061] running on GCE node
I20260709 12:13:11.500325 23926 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:11.505060 24713 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:11.513798 24988 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.474517 24939 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.504061 25001 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:11.503989 25019 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.524206 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.508594 25038 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.508745 24922 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.528489 23784 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.525755 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.533202 23783 raft_consensus.cc:2368] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4.
I20260709 12:13:11.505126 24994 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.513432 25002 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.503270 25011 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.548293 23784 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:11.518821 24934 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:11.564129 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 253726us
I20260709 12:13:11.539283 24994 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.568075 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.569715 23851 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.571529 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.558107 23780 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:11.536423 25038 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.579036 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.583964 25007 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.584589 25007 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.586743 25007 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.588517 24885 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:11.589937 24885 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:11.590418 24885 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:11.532191 25019 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.593276 25019 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.572955 24855 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:13:11.593829 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.595235 24982 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.569283 24926 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.596073 24932 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.598122 24994 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.601553 25002 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.602833 24934 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.603194 24939 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.576387 24971 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.604141 24971 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.604480 24971 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.605736 25038 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.607198 22629 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:13:11.607532 22629 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:13:11.607775 22629 hybrid_clock.cc:648] HybridClock initialized: now 1783599191607758 us; error 0 us; skew 500 ppm
W20260709 12:13:11.581398 25035 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:13:11.636001 22629 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:13:11.591228 24885 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.674785 24987 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.675511 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:11.676589 25039 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:13:11.684531 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.685199 23929 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:11.686561 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.687222 23929 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:11.689782 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.691529 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.693081 24994 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.693538 24994 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.693915 24994 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.683789 25011 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.642861 24713 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.692577 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.697999 24984 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.712687 23931 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:13:11.700719 25011 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.707741 24931 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:11.722080 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 144962us
I20260709 12:13:11.707619 24971 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.718129 25011 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.731040 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.732188 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.739013 23858 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:11.735944 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.740043 23856 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:11.744277 23900 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.745342 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.727919 24934 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.748248 23929 raft_consensus.cc:2368] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:11.718341 24984 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.741575 25011 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.762892 24932 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:11.763392 24932 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.763751 24932 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.754082 24855 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:11.765558 24855 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.765939 24855 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.766222 24922 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.744848 24594 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:11.772083 25045 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:11.753700 24934 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.778828 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.779729 23782 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:11.779572 25011 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.774950 24855 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.782320 24713 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.782720 25007 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:11.783119 24855 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.783224 25007 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.727180 24931 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.785532 25007 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.786062 24931 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.790670 24932 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.792187 23932 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:11.775367 24934 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.754964 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.791478 24971 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.747262 25036 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.794768 25011 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.813653 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.794533 25009 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.817957 24934 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.818660 24934 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.819661 25009 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.820006 23851 raft_consensus.cc:2368] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.822983 24984 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.824564 24932 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:11.826584 23853 raft_consensus.cc:2410] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:11.831240 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.832939 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.833241 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.834489 23853 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:11.834631 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.840353 24995 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.841336 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.842199 23925 raft_consensus.cc:2368] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:11.839131 25036 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.848513 25036 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.850418 24934 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.851759 25011 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.852309 25011 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.853113 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 98) took 1538 ms. Trace:
I20260709 12:13:11.853987 24931 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:11.854851 25011 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.854662 24931 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.855832 24994 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:11.853464 23681 rpcz_store.cc:276] 0709 12:13:10.317047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.317535 (+   488us) service_pool.cc:224] Handling call
0709 12:13:11.848074 (+1530539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:10.779668 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:10.780295 (+   627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:10.780333 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:13:10.780364 (+    31us) tablet.cc:662] Decoding operations
0709 12:13:10.780712 (+   348us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:10.780789 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:10.780819 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:10.781022 (+   203us) tablet.cc:701] Row locks acquired
0709 12:13:10.781041 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:13:10.781196 (+   155us) write_op.cc:270] Start()
0709 12:13:10.781308 (+   112us) write_op.cc:276] Timestamp: P: 1783599190781139 usec, L: 0
0709 12:13:10.781330 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:13:10.782133 (+   803us) log.cc:844] Serialized 537 byte log entry
0709 12:13:11.022907 (+240774us) op_driver.cc:464] REPLICATION: finished
0709 12:13:11.208843 (+185936us) write_op.cc:301] APPLY: starting
0709 12:13:11.212574 (+  3731us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:11.219519 (+  6945us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:11.241666 (+ 22147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:11.241878 (+   212us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:11.242025 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:11.242067 (+    42us) write_op.cc:312] APPLY: finished
0709 12:13:11.242259 (+   192us) log.cc:844] Serialized 22 byte log entry
0709 12:13:11.242408 (+   149us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:11.242475 (+    67us) write_op.cc:454] Released schema lock
0709 12:13:11.242506 (+    31us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:11.242853 (+   347us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:11.242883 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":178161,"apply.run_cpu_time_us":1918,"apply.run_wall_time_us":41057,"num_ops":1,"prepare.queue_time_us":181354,"prepare.run_cpu_time_us":2793,"prepare.run_wall_time_us":2793,"replication_time_us":241431,"thread_start_us":337860,"threads_started":2}]]}
I20260709 12:13:11.861155 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.859627 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.862046 23776 raft_consensus.cc:2368] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:11.862854 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.863504 23925 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:11.861644 24931 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.865324 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.865792 23749 leader_election.cc:400] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:11.866094 23749 leader_election.cc:403] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.866614 24931 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.866760 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.856877 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.867703 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.868323 23785 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.868402 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.868981 23934 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.861344 24994 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:11.901199 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.876639 23928 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:13:11.877241 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 131967us
I20260709 12:13:11.879041 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.880319 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.882128 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.883203 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:11.905866 23777 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:11.905007 23776 raft_consensus.cc:2410] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:13:11.908936 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.909565 23858 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.910315 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:11.910859 23852 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.911509 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.912055 23852 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.913746 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.914408 23860 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:11.917999 23900 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.905071 24934 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.926787 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:11.933486 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.934103 23785 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:11.935958 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:11.936597 23778 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.928220 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:11.929237 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.930348 24931 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:11.950102 23783 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:11.927739 24984 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:11.950804 24984 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.951206 24984 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:11.907761 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.956573 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.957809 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.958949 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:11.961714 23856 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:11.963405 24926 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:11.934129 23858 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:11.943109 23934 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:11.944555 23931 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:11.904937 24937 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:13:11.932101 24934 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.987082 23900 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.980229 23860 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:11.942287 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:11.980846 23852 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:11.980808 23929 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:11.983151 23785 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:12.003810 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61311.7us
I20260709 12:13:12.006070 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:12.002650 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.006871 23785 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:12.007652 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.008440 25056 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:12.008951 25056 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.005318 24926 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:12.009783 25056 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.011981 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.013262 23749 leader_election.cc:400] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.013593 23749 leader_election.cc:403] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.024194 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.023873 24926 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.026932 24926 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.029407 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.023450 23855 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:11.996703 24934 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:12.032091 23900 leader_election.cc:400] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:12.036309 23900 leader_election.cc:403] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.038551 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.035305 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.042622 23776 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 2: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:12.023981 24925 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.040355 24982 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:12.045204 25042 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:12.053401 23899 leader_election.cc:400] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:12.037487 25055 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:12.037238 25001 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.036933 24971 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.058578 22629 webserver.cc:533] Webserver started at http://127.22.25.68:39541/ using document root <none> and password file <none>
I20260709 12:13:12.051643 25042 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.091784 24984 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.093072 23930 raft_consensus.cc:2368] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.094063 24984 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:12.095175 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.096009 23855 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:12.099398 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 639) took 1084 ms. Trace:
I20260709 12:13:12.099869 23933 rpcz_store.cc:276] 0709 12:13:11.015140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.015895 (+   755us) service_pool.cc:224] Handling call
0709 12:13:11.016187 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:11.027489 (+ 11302us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:11.517618 (+490129us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b18001a14c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf 00007fb2ad7cc038
0709 12:13:11.623759 (+106141us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.742596 (+118837us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:11.743156 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:13:11.743294 (+   138us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:11.743326 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.099018 (+355692us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b54000e4054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:12.099079 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.099130 (+    51us) raft_consensus.cc:1685] finished
0709 12:13:12.099144 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.099326 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37873,"prepare.queue_time_us":203169,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":16963,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":15867,"spinlock_wait_cycles":37798528,"thread_start_us":213827,"threads_started":2,"wal-append.queue_time_us":188260}
I20260709 12:13:12.095155 24925 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.107795 25042 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.108436 24934 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.109550 23784 raft_consensus.cc:2368] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:12.109748 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.110819 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.110819 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.114317 23855 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:12.115751 23900 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.116351 25018 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:13:12.119473 25069 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:12.120177 25069 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.121250 25069 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:12.122713 25064 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
W20260709 12:13:12.125207 25066 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:13:12.126958 24936 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:12.127606 24936 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.127846 25055 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.128341 25055 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:12.136782 25027 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:12.137272 25027 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:12.137645 25027 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:12.140728 25052 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:12.141171 25052 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.141985 25052 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.142608 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.143146 25052 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.145067 23749 leader_election.cc:400] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.145427 23749 leader_election.cc:403] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.133978 22629 fs_manager.cc:362] Metadata directory not provided
I20260709 12:13:12.127871 25012 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:12.152913 23825 leader_election.cc:400] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4.
I20260709 12:13:12.154712 23825 leader_election.cc:403] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.153919 23782 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:12.152091 22629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:13:12.160663 23825 leader_election.cc:400] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:12.165066 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:12.166109 23785 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:12.168151 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:12.175639 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.176750 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:12.178097 23779 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:12.147316 25007 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:12.187613 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.151865 24925 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:12.190765 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.191500 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:12.193670 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 125325us
I20260709 12:13:12.239954 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.242784 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.186415 25007 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.240736 22629 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:13:12.243496 23859 raft_consensus.cc:2368] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.193886 25011 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.248000 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:12.252092 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 61409.5us
I20260709 12:13:12.190099 25012 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.128443 24936 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.264524 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.268759 23859 raft_consensus.cc:2368] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.254931 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.260901 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:12.258608 23856 raft_consensus.cc:2368] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.178229 24807 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:13:12.273567 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.252108 24971 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.255165 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.249781 25007 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.279317 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:12.266568 25059 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:13:12.294065 25012 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.296756 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.297557 23860 raft_consensus.cc:2368] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.282130 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.298806 23856 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:12.253192 24931 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.285943 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.286365 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.286935 25050 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4.
I20260709 12:13:12.301302 23899 leader_election.cc:400] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:12.274946 25011 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.316470 25055 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.318773 25055 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.304644 25074 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.312705 24994 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:12.324972 24994 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.327220 24994 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.381384 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.389091 23777 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:12.393757 23854 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:12.301630 25001 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:12.341902 25070 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:13:12.401088 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:12.401979 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:12.409070 23932 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:12.328503 24934 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.407660 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:12.412245 23781 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:12.403122 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 148266us
I20260709 12:13:12.403834 23825 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.337811 22629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-3-root/instance:
uuid: "05679ef5e2b74861aad2a08b6a05e276"
format_stamp: "Formatted at 2026-07-09 12:13:12 on dist-test-slave-sc22"
I20260709 12:13:12.329327 23778 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:12.419226 25001 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.327514 25027 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.424959 25027 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.328308 24971 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.429225 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.430756 24643 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:12.431396 24643 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:12.435307 24986 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:13:12.416988 23751 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:12.440428 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 160131us
I20260709 12:13:12.454957 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:12.455490 23751 leader_election.cc:400] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.456625 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.463523 23751 leader_election.cc:400] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.469105 23858 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:12.472374 23781 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:12.474077 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:12.474831 23855 raft_consensus.cc:2368] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.487700 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:12.466177 23751 leader_election.cc:400] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:13:12.492085 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:12.493201 23932 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:12.495043 23751 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.515560 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:12.515825 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43984.6us
I20260709 12:13:12.502269 24994 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.533398 23751 leader_election.cc:400] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
W20260709 12:13:12.537832 25092 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:13:12.473428 24931 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.536970 25083 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.538297 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.541174 23751 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.542124 23751 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.525929 24807 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:12.553004 24925 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.556707 23785 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:12.553785 24931 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.560320 22629 fs_manager.cc:696] Time spent creating directory manager: real 0.143s	user 0.011s	sys 0.016s
I20260709 12:13:12.548647 25011 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:12.564731 25084 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:12.547961 24994 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:12.569295 23825 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.576843 23751 leader_election.cc:400] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:12.578914 25043 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:12.579380 25043 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.582845 25051 raft_consensus.cc:2764] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:12.584139 24807 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:12.585088 24807 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.597214 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.599540 23785 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:12.605661 23779 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:12.606978 25077 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:12.607450 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:12.609968 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.612703 23851 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:12.614326 23784 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:12.613473 24994 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.565376 25084 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.617797 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:12.622192 25081 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:13:12.618661 24982 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.580183 25043 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.613304 25038 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:13:12.636638 24855 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.630999 25000 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:12.639189 24982 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.639967 25007 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.640196 24982 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.641060 23781 raft_consensus.cc:2368] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:12.644318 25007 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.645896 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.646308 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:12.645710 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:12.646894 23932 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.649781 25084 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.654497 23932 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:12.656023 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:12.656046 23825 leader_election.cc:400] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:12.661765 23853 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:12.662623 23825 leader_election.cc:403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.663379 24994 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:12.664098 25037 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.668828 24988 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:12.677171 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:12.680445 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:12.699016 23777 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:12.702991 23785 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:12.712360 25095 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:12.713016 25095 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:12.679482 23825 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:12.723135 25100 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260709 12:13:12.731048 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51677.7us
I20260709 12:13:12.738454 23858 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:12.778777 22629 fs_manager.cc:730] Time spent opening block manager: real 0.174s	user 0.020s	sys 0.004s
W20260709 12:13:12.780241 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55323.3us
I20260709 12:13:12.779341 22629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-3-root
uuid: "05679ef5e2b74861aad2a08b6a05e276"
format_stamp: "Formatted at 2026-07-09 12:13:12 on dist-test-slave-sc22"
I20260709 12:13:12.735379 25084 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:12.791041 22629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:13:12.797065 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59052.4us
I20260709 12:13:12.800556 23858 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:12.795424 25000 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:13:12.846109 24925 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:12.870844 24925 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.873391 24925 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.874470 24855 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:13:12.878266 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:12.866811 24934 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:12.880014 24934 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.882265 24934 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:12.883013 25043 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:12.885039 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:12.794520 25001 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:13:12.906000 24939 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:12.906517 24939 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:12.907459 24939 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.908845 24939 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:12.846637 25040 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:13:12.890934 24934 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:12.830315 25095 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.929991 25037 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:12.934494 24934 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:12.965587 23783 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:12.990794 23851 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:12.989396 23857 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:12.993014 23784 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:13.000638 24713 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.005223 24934 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:13.015909 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:13.018848 23779 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:13.022378 23930 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:13.022410 23927 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:13.030337 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:13.031139 23858 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:13.031757 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:13.031222 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:13.045269 24982 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:13:13.033298 24984 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:13:13.077901 24789 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:13.078884 25018 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.080360 23751 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:13.081465 23926 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:13.081516 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:13.097244 24994 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:13.099416 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52191.8us
I20260709 12:13:13.097338 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:13.098677 23853 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:13.102603 24934 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:13.107241 24931 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:13:13.128332 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 77278.4us
I20260709 12:13:13.123387 25012 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:13:13.158883 23781 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:13.174983 25088 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:13:13.199187 24852 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:13.189317 24994 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:13.217689 24982 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:13.218551 23926 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:13:13.218952 23926 pending_rounds.cc:85] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 1
I20260709 12:13:13.219201 23926 pending_rounds.cc:107] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:13:13.225668 23926 raft_consensus.cc:2889] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:13.266310 23777 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:13.281955 23856 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:13.287496 24713 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:13:13.295096 24852 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:13.343822 22629 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:13:13.359762 23777 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:13.359484 25012 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:13:13.366669 23680 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.374481 23680 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.378207 23680 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.393285 23680 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.405122 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 114) took 1211 ms. Trace:
I20260709 12:13:13.410578 23683 rpcz_store.cc:276] 0709 12:13:12.193582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.193732 (+   150us) service_pool.cc:224] Handling call
0709 12:13:13.405039 (+1211307us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":42}
I20260709 12:13:13.456350 24986 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:13:13.460068 22629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:13:13.510989 23854 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:13.515628 25011 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.538369 24926 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:13.538908 24926 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:13.541760 24926 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:13.543958 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:13.544644 23856 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:13.545111 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:13.545933 23780 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:13.548909 23900 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:13.550406 24885 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:13.552404 23928 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:13.551111 24885 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:13.553721 24885 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:13.554164 24934 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:13.630450 22629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:13:13.702754 24885 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:13.717785 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:13.727247 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:13.728097 23784 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:13.729907 22629 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:13:13.730257 24885 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:13.739161 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:13.730281 22629 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:13:13.740549 22629 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:13:13.740813 22629 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:13:13.744146 23900 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:13.745412 24971 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:13.745929 24971 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:13.746760 24971 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:13.808383 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 99) took 1463 ms. Trace:
I20260709 12:13:13.827673 23680 rpcz_store.cc:276] 0709 12:13:12.344578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.344717 (+   139us) service_pool.cc:224] Handling call
0709 12:13:13.808299 (+1463582us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:13.560384 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:13.563730 (+  3346us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:13.563761 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:13:13.563780 (+    19us) tablet.cc:662] Decoding operations
0709 12:13:13.564360 (+   580us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:13.564432 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:13.564464 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:13:13.564830 (+   366us) tablet.cc:701] Row locks acquired
0709 12:13:13.564847 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:13:13.564970 (+   123us) write_op.cc:270] Start()
0709 12:13:13.565083 (+   113us) write_op.cc:276] Timestamp: P: 1783599193564935 usec, L: 0
0709 12:13:13.565106 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:13:13.616388 (+ 51282us) log.cc:844] Serialized 1693 byte log entry
0709 12:13:13.734064 (+117676us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b180013a680. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:13:13.742101 (+  8037us) op_driver.cc:464] REPLICATION: finished
0709 12:13:13.753057 (+ 10956us) write_op.cc:301] APPLY: starting
0709 12:13:13.753148 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:13.753302 (+   154us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:13.753318 (+    16us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:13.755838 (+  2520us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:13.772399 (+ 16561us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:13:13.772458 (+    59us) write_op.cc:312] APPLY: finished
0709 12:13:13.772729 (+   271us) log.cc:844] Serialized 40 byte log entry
0709 12:13:13.773464 (+   735us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:13.773573 (+   109us) write_op.cc:454] Released schema lock
0709 12:13:13.773612 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:13.820440 (+ 46828us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:13.820475 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103611,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":10684,"apply.run_cpu_time_us":2307,"apply.run_wall_time_us":67335,"num_ops":4,"prepare.queue_time_us":74480,"prepare.run_cpu_time_us":73775,"prepare.run_wall_time_us":252472,"raft.queue_time_us":6114,"raft.run_cpu_time_us":7124,"raft.run_wall_time_us":12409,"replication_time_us":176826,"spinlock_wait_cycles":3541376,"thread_start_us":155236,"threads_started":4,"wal-append.queue_time_us":112615}]]}
I20260709 12:13:13.837936 25011 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.838560 25011 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:13.840791 25011 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:13.849800 23683 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.852648 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.853605 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:13.856249 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:13.857390 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:13.858292 25011 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:13.858690 25011 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.859025 25011 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:13.866760 23683 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.868362 23683 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.872180 23683 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:13.873821 23683 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.875674 23683 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.877102 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.880223 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 105) took 1686 ms. Trace:
I20260709 12:13:13.880770 23681 rpcz_store.cc:276] 0709 12:13:12.244271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.253310 (+  9039us) service_pool.cc:224] Handling call
0709 12:13:13.880146 (+1626836us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60683,"reported_tablets":21}
I20260709 12:13:13.909441 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:13.877526 25011 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:13.930428 23683 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.939239 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:13.941146 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:13.942533 23857 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:13.948415 25011 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:13.961982 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:14.070842 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 51227.9us
I20260709 12:13:14.071178 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:14.072845 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:14.074095 24931 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:14.074853 24931 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:14.076156 24931 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:14.113227 23683 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:14.171131 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:14.269501 23858 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:14.278143 24932 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:13:14.295930 24932 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:14.296372 24932 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:14.297196 24932 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:14.298378 24932 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:14.302655 23784 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:14.628011 23857 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:14.632652 24931 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:14.684460 23928 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:13:14.684937 23928 pending_rounds.cc:85] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 4
I20260709 12:13:14.685245 23928 pending_rounds.cc:107] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:13:14.685600 23928 raft_consensus.cc:2889] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:14.727946 25011 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:14.911659 25207 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.22.25.68:43683 every 8 connection(s)
I20260709 12:13:14.911967 22629 rpc_server.cc:307] RPC server started. Bound to: 127.22.25.68:43683
I20260709 12:13:14.946002 22629 auto_leader_rebalancer-test.cc:439] add a tserver: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:14.959961 25208 heartbeater.cc:344] Connected to a master server at 127.22.25.126:39113
I20260709 12:13:14.960594 25208 heartbeater.cc:461] Registering TS with master...
I20260709 12:13:14.962800 25208 heartbeater.cc:507] Master 127.22.25.126:39113 requested a full tablet report, sending...
I20260709 12:13:14.965471 23683 ts_manager.cc:194] Registered new tserver with Master: 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:14.967445 23683 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50498
I20260709 12:13:15.483053 23857 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:15.529789 23784 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.531479 25006 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.547751 23932 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.552327 25162 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.557515 23779 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.569811 25051 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.587251 23683 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.603657 23932 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.639284 23925 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:15.673619 23824 consensus_peers.cc:597] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d79b6f8bf5274da7a3f667b9988c5a84. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.683424 23857 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:15.683811 23777 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:15.686776 25153 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:15.688222 25042 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:15.712915 25153 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.718689 23777 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.740767 23925 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:15.758502 23857 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.760071 23680 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:15.773694 23898 consensus_peers.cc:597] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8368f2e695204683affed6a9bd84ba96. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:15.810847 23898 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 872245a7005f472bb7f6cfafda680b74. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.811268 23858 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:15.812225 23777 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:15.813040 25042 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:15.814158 24982 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:15.850373 25153 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.852891 23858 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.858353 23776 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.921370 23680 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:15.946806 23777 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:15.970703 25208 heartbeater.cc:499] Master 127.22.25.126:39113 was elected leader, sending a full tablet report...
I20260709 12:13:15.976339 23925 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:15.978048 24931 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:15.989675 23858 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:15.991159 24931 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:13:15.999732 23749 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20fc63afa03b45c0bf3f58aae82f3e3d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:16.015734 23925 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:16.013711 25236 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:16.056596 23854 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:16.093953 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 6, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:16.101344 25238 ts_tablet_manager.cc:933] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:16.118345 25238 tablet_copy_client.cc:323] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:16.158761 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet d79b6f8bf5274da7a3f667b9988c5a84 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:16.159453 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet d79b6f8bf5274da7a3f667b9988c5a84 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-d79b6f8bf5274da7a3f667b9988c5a84
I20260709 12:13:16.168663 23870 tablet_copy_source_session.cc:215] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.174542 25238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d79b6f8bf5274da7a3f667b9988c5a84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:13:16.233381 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54285.5us
I20260709 12:13:16.261658 25242 ts_tablet_manager.cc:933] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:16.294178 25242 tablet_copy_client.cc:323] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:16.314735 25238 tablet_copy_client.cc:806] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.315425 25238 tablet_copy_client.cc:670] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.325384 25238 tablet_copy_client.cc:538] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.352348 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 8368f2e695204683affed6a9bd84ba96 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:16.353057 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 8368f2e695204683affed6a9bd84ba96 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-8368f2e695204683affed6a9bd84ba96
I20260709 12:13:16.353350 25245 ts_tablet_manager.cc:933] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:16.352483 25238 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:16.362685 23944 tablet_copy_source_session.cc:215] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.381551 25242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8368f2e695204683affed6a9bd84ba96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.392402 25245 tablet_copy_client.cc:323] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:16.400656 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 872245a7005f472bb7f6cfafda680b74 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:16.401224 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 872245a7005f472bb7f6cfafda680b74 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-872245a7005f472bb7f6cfafda680b74
I20260709 12:13:16.408394 25242 tablet_copy_client.cc:806] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.409020 25242 tablet_copy_client.cc:670] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.409997 23944 tablet_copy_source_session.cc:215] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.419981 25245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872245a7005f472bb7f6cfafda680b74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.426836 25248 ts_tablet_manager.cc:933] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:16.434945 25242 tablet_copy_client.cc:538] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.441330 25238 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.442396 25238 tablet_bootstrap.cc:492] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:16.447355 25245 tablet_copy_client.cc:806] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.447978 25245 tablet_copy_client.cc:670] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.448612 25238 ts_tablet_manager.cc:1403] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.096s	user 0.027s	sys 0.004s
I20260709 12:13:16.449573 25248 tablet_copy_client.cc:323] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:13:16.454527 25245 tablet_copy_client.cc:538] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.463071 25238 raft_consensus.cc:359] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.464058 25238 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:16.464797 25238 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.469475 25245 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:16.485271 25238 ts_tablet_manager.cc:1434] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.002s
I20260709 12:13:16.489154 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d79b6f8bf5274da7a3f667b9988c5a84 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:16.489724 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d79b6f8bf5274da7a3f667b9988c5a84 on tablet d79b6f8bf5274da7a3f667b9988c5a84 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:16.513895 25242 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:16.531021 25245 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.531435 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet 20fc63afa03b45c0bf3f58aae82f3e3d from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:13:16.532069 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet 20fc63afa03b45c0bf3f58aae82f3e3d from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-20fc63afa03b45c0bf3f58aae82f3e3d
I20260709 12:13:16.544750 25245 tablet_bootstrap.cc:492] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:16.545563 25245 ts_tablet_manager.cc:1403] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.009s
I20260709 12:13:16.548666 25245 raft_consensus.cc:359] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.549495 25245 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:16.550110 25245 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.567889 25245 ts_tablet_manager.cc:1434] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 12:13:16.572237 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-872245a7005f472bb7f6cfafda680b74 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:16.572687 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-872245a7005f472bb7f6cfafda680b74 on tablet 872245a7005f472bb7f6cfafda680b74 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:16.627576 25177 raft_consensus.cc:1217] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:16.628554 25242 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.629519 25242 tablet_bootstrap.cc:492] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:16.630213 25242 ts_tablet_manager.cc:1403] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.117s	user 0.018s	sys 0.012s
I20260709 12:13:16.634550 25242 raft_consensus.cc:359] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.635388 25242 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:16.636073 25242 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:16.649999 25242 ts_tablet_manager.cc:1434] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 12:13:16.651974 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8368f2e695204683affed6a9bd84ba96 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:16.652437 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8368f2e695204683affed6a9bd84ba96 on tablet 8368f2e695204683affed6a9bd84ba96 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:16.700310 23795 tablet_copy_source_session.cc:215] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.721439 25248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20fc63afa03b45c0bf3f58aae82f3e3d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:13:16.728057 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 89678.1us
I20260709 12:13:16.751642 25177 raft_consensus.cc:1217] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:16.758450 25248 tablet_copy_client.cc:806] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.759052 25248 tablet_copy_client.cc:670] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.764916 25248 tablet_copy_client.cc:538] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.794348 25248 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:16.846295 25177 raft_consensus.cc:1217] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:17.102985 25248 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:17.104552 25248 tablet_bootstrap.cc:492] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:17.105300 25248 ts_tablet_manager.cc:1403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.312s	user 0.031s	sys 0.011s
I20260709 12:13:17.110227 25248 raft_consensus.cc:359] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:17.111111 25248 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:17.111809 25248 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:17.126266 25248 ts_tablet_manager.cc:1434] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:13:17.134812 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-20fc63afa03b45c0bf3f58aae82f3e3d received from {username='slave'} at 127.0.0.1:40134
I20260709 12:13:17.135351 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-20fc63afa03b45c0bf3f58aae82f3e3d on tablet 20fc63afa03b45c0bf3f58aae82f3e3d with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:17.235534 25265 raft_consensus.cc:1064] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:17.238657 25265 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:17.257965 24807 raft_consensus.cc:1064] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:17.261647 24807 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:17.275763 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52260.9us
I20260709 12:13:17.294468 25177 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.296118 23928 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.296384 25162 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:17.299209 25255 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.326715 23779 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.327535 25261 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.328612 23928 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.346340 25177 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:17.350055 25257 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.358057 23856 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:17.359999 23784 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:17.360893 25259 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:17.361812 25153 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.390841 23681 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.413543 25255 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:17.420414 25257 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.426283 23784 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.432989 23856 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.442975 25177 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.462661 23784 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.468875 25153 raft_consensus.cc:1064] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:17.477147 25153 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:17.551067 23683 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.587230 25177 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:17.602720 25256 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:17.607797 25177 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.617775 23854 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:17.618135 23784 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:17.622975 25175 raft_consensus.cc:1217] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEARNER]: Deduplicated request from leader. Original: 6.5->[6.6-6.6]   Dedup: 6.6->[]
I20260709 12:13:17.629349 25259 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.648983 25256 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.666802 25259 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.688256 23858 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.712337 23777 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.734227 25176 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.786557 23683 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.846869 23928 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:17.856500 25176 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:17.858785 25257 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:17.867326 24982 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.870060 25176 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.879683 23854 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:17.889003 25256 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:17.890463 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8368f2e695204683affed6a9bd84ba96 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:17.899540 23858 raft_consensus.cc:2955] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.907907 23681 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.923687 23928 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:17.928205 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 8368f2e695204683affed6a9bd84ba96 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:17.931046 25176 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:17.935729 23780 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:17.937916 25259 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:17.940261 25256 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:17.951164 24982 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.972788 25177 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:17.984025 23780 raft_consensus.cc:2955] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.006292 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 872245a7005f472bb7f6cfafda680b74 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:18.012084 23682 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:18.020828 23858 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:18.028304 23777 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:18.029724 25162 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:18.036195 24807 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.041254 25177 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:18.042587 25261 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:18.043437 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 872245a7005f472bb7f6cfafda680b74 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:18.040958 23777 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.069386 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d79b6f8bf5274da7a3f667b9988c5a84 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:18.069720 25287 tablet_replica.cc:333] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:18.070962 25287 raft_consensus.cc:2243] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:18.083083 25287 raft_consensus.cc:2272] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:18.084903 25288 tablet_replica.cc:333] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:18.086556 25288 raft_consensus.cc:2243] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:18.092597 23681 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 1 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.103760 25288 raft_consensus.cc:2272] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:18.087859 25177 raft_consensus.cc:2955] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.124980 25287 ts_tablet_manager.cc:1916] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:18.140839 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet d79b6f8bf5274da7a3f667b9988c5a84 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:18.172433 25288 ts_tablet_manager.cc:1916] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:18.223464 25288 ts_tablet_manager.cc:1929] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:18.223958 25288 log.cc:1199] T 872245a7005f472bb7f6cfafda680b74 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/872245a7005f472bb7f6cfafda680b74
I20260709 12:13:18.225754 25290 tablet_replica.cc:333] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:18.225683 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 872245a7005f472bb7f6cfafda680b74 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:18.227067 25290 raft_consensus.cc:2243] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:18.227772 25290 raft_consensus.cc:2272] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:18.241540 25289 raft_consensus.cc:1064] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:18.244328 25289 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:18.239868 25290 ts_tablet_manager.cc:1916] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:18.258359 25287 ts_tablet_manager.cc:1929] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:18.258823 25287 log.cc:1199] T 8368f2e695204683affed6a9bd84ba96 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/8368f2e695204683affed6a9bd84ba96
I20260709 12:13:18.268764 25177 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:18.269910 23858 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:18.271353 25289 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:18.275785 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 8368f2e695204683affed6a9bd84ba96 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:18.268810 23928 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:18.278179 25278 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:18.279102 25289 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:18.305414 25278 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.307343 25290 ts_tablet_manager.cc:1929] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:18.306649 25177 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.315416 23858 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.322748 23928 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.307807 25290 log.cc:1199] T d79b6f8bf5274da7a3f667b9988c5a84 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/d79b6f8bf5274da7a3f667b9988c5a84
I20260709 12:13:18.330130 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet d79b6f8bf5274da7a3f667b9988c5a84 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:18.334349 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 6 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:18.351279 23780 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:18.378629 25177 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:13:18.379370 23858 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:13:18.381150 25278 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:18.382378 25280 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:18.423229 25280 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.462378 23856 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.481237 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 20fc63afa03b45c0bf3f58aae82f3e3d with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:18.481037 25177 raft_consensus.cc:2955] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Committing config change with OpId 6.8: config changed from index 7 to 8, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:18.497243 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 7 to 8, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:18.520728 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 20fc63afa03b45c0bf3f58aae82f3e3d with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:18.521783 25290 tablet_replica.cc:333] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:18.522719 25290 raft_consensus.cc:2243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:18.523582 25290 raft_consensus.cc:2272] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:18.533126 25290 ts_tablet_manager.cc:1916] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:18.554839 25290 ts_tablet_manager.cc:1929] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.7
I20260709 12:13:18.555243 25290 log.cc:1199] T 20fc63afa03b45c0bf3f58aae82f3e3d P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/20fc63afa03b45c0bf3f58aae82f3e3d
I20260709 12:13:18.557087 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 20fc63afa03b45c0bf3f58aae82f3e3d (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:19.679466 23856 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:19.690104 23824 consensus_peers.cc:597] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 526cc198da134bb590358ed73c23d59e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:19.691963 23932 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:19.693531 25261 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:19.698827 23780 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:19.700220 25162 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:19.719679 25261 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.725225 23928 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.772158 23683 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.773468 23928 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:19.831383 23777 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
W20260709 12:13:19.842118 23898 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc9ef29919aa42aaa2b7545c10e39eff. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:19.842486 23779 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:19.849074 23856 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:19.850126 25256 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:19.854059 25297 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:19.879330 25256 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.882910 23856 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.930990 23779 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:19.940577 23777 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:19.960064 23682 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:19.975170 23857 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:19.970713 23749 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bec362a6d01840d39be46f0c3d28c218. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:19.970721 23928 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:19.978051 25305 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:19.981132 25280 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:20.034101 25305 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.040858 23928 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.043349 23858 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.063827 23682 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:20.064076 23856 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.133791 23932 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:20.141034 23779 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:20.145726 24807 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:20.151834 23824 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 24dc0370e122479081184511cb1224f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:20.154287 25314 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:20.176486 25301 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.180899 23780 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.212111 23932 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.220775 23682 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:20.280715 25317 ts_tablet_manager.cc:933] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:20.309412 25319 ts_tablet_manager.cc:933] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:20.327792 25317 tablet_copy_client.cc:323] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:20.329777 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 526cc198da134bb590358ed73c23d59e from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:20.330338 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 526cc198da134bb590358ed73c23d59e from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-526cc198da134bb590358ed73c23d59e
I20260709 12:13:20.338459 23870 tablet_copy_source_session.cc:215] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:20.342087 25317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 526cc198da134bb590358ed73c23d59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:20.352494 25319 tablet_copy_client.cc:323] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:20.356348 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet cc9ef29919aa42aaa2b7545c10e39eff from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:20.356920 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet cc9ef29919aa42aaa2b7545c10e39eff from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-cc9ef29919aa42aaa2b7545c10e39eff
I20260709 12:13:20.361003 25317 tablet_copy_client.cc:806] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:20.361593 25317 tablet_copy_client.cc:670] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:20.365810 25317 tablet_copy_client.cc:538] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:20.369755 23944 tablet_copy_source_session.cc:215] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:20.374341 25319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9ef29919aa42aaa2b7545c10e39eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:20.380895 25317 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:20.410063 25317 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:20.411060 25317 tablet_bootstrap.cc:492] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:20.411829 25317 ts_tablet_manager.cc:1403] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.019s
I20260709 12:13:20.415398 25317 raft_consensus.cc:359] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.416296 25317 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:20.417106 25317 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.421772 25319 tablet_copy_client.cc:806] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:20.422489 25319 tablet_copy_client.cc:670] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:20.430847 25319 tablet_copy_client.cc:538] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:20.453822 25319 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:20.457068 25317 ts_tablet_manager.cc:1434] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.002s
I20260709 12:13:20.461629 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-526cc198da134bb590358ed73c23d59e received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:20.462086 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-526cc198da134bb590358ed73c23d59e on tablet 526cc198da134bb590358ed73c23d59e with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:20.472536 25317 ts_tablet_manager.cc:933] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:20.487682 25317 tablet_copy_client.cc:323] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:13:20.489607 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet bec362a6d01840d39be46f0c3d28c218 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:13:20.490166 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet bec362a6d01840d39be46f0c3d28c218 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-bec362a6d01840d39be46f0c3d28c218
I20260709 12:13:20.500401 23795 tablet_copy_source_session.cc:215] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:20.511209 25317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bec362a6d01840d39be46f0c3d28c218. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:20.533699 25319 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:20.540108 25319 tablet_bootstrap.cc:492] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:20.539013 25317 tablet_copy_client.cc:806] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:20.540951 25317 tablet_copy_client.cc:670] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:20.542840 25319 ts_tablet_manager.cc:1403] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.089s	user 0.033s	sys 0.011s
I20260709 12:13:20.545133 25317 tablet_copy_client.cc:538] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:20.557219 25319 raft_consensus.cc:359] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.558310 25319 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:20.559033 25319 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.567227 25319 ts_tablet_manager.cc:1434] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:13:20.555552 25317 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:20.571813 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-cc9ef29919aa42aaa2b7545c10e39eff received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:20.572292 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-cc9ef29919aa42aaa2b7545c10e39eff on tablet cc9ef29919aa42aaa2b7545c10e39eff with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:20.608783 25317 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:20.609869 25317 tablet_bootstrap.cc:492] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:20.610646 25317 ts_tablet_manager.cc:1403] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.017s
I20260709 12:13:20.614624 25317 raft_consensus.cc:359] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.615623 25317 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:20.616387 25317 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.625681 25317 ts_tablet_manager.cc:1434] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:13:20.634299 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-bec362a6d01840d39be46f0c3d28c218 received from {username='slave'} at 127.0.0.1:40134
I20260709 12:13:20.634760 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-bec362a6d01840d39be46f0c3d28c218 on tablet bec362a6d01840d39be46f0c3d28c218 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:20.710287 25176 raft_consensus.cc:1217] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 12:13:20.746881 25317 ts_tablet_manager.cc:933] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:20.761452 25317 tablet_copy_client.cc:323] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:20.763290 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 24dc0370e122479081184511cb1224f1 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:20.763908 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 24dc0370e122479081184511cb1224f1 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-24dc0370e122479081184511cb1224f1
I20260709 12:13:20.777125 23870 tablet_copy_source_session.cc:215] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:20.782387 25317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0370e122479081184511cb1224f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:20.806775 25317 tablet_copy_client.cc:806] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:20.807418 25317 tablet_copy_client.cc:670] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:20.811456 25317 tablet_copy_client.cc:538] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:20.818953 25176 raft_consensus.cc:1217] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:20.824810 25176 raft_consensus.cc:1217] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:20.828526 25317 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:20.907084 25317 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:20.908160 25317 tablet_bootstrap.cc:492] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:20.908943 25317 ts_tablet_manager.cc:1403] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.081s	user 0.039s	sys 0.007s
I20260709 12:13:20.911983 25317 raft_consensus.cc:359] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.912868 25317 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:20.913529 25317 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:20.920781 25317 ts_tablet_manager.cc:1434] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:13:20.924594 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-24dc0370e122479081184511cb1224f1 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:20.925065 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-24dc0370e122479081184511cb1224f1 on tablet 24dc0370e122479081184511cb1224f1 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:21.110877 25297 raft_consensus.cc:1064] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:21.114771 25297 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 2, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.218060 25261 raft_consensus.cc:1064] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:21.220481 25261 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.292558 23928 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:21.303709 23779 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:13:21.332589 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51892.3us
I20260709 12:13:21.359745 25261 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:21.359809 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55524.5us
I20260709 12:13:21.362354 25176 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:21.362759 25177 raft_consensus.cc:1217] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:13:21.368525 25177 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:21.370462 25305 raft_consensus.cc:1064] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:21.373263 25305 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.373706 25297 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:13:21.380025 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82333.1us
I20260709 12:13:21.386179 25334 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.389041 25334 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:21.410477 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 117770us
I20260709 12:13:21.422144 23854 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:13:21.429674 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47808.6us
I20260709 12:13:21.470577 23777 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:13:21.471525 25261 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.473420 25176 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.476836 25153 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.491389 23681 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.493918 25296 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.497255 25177 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:21.501263 25344 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:21.507274 23858 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:21.508507 25344 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:21.511031 23928 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:21.514233 23781 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.523655 25344 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:21.535549 23928 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.578585 23854 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.578256 25344 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.579386 23856 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.586306 25337 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.602614 23928 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.602885 25177 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.615345 25176 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.649196 23776 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.679427 23680 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.689823 23856 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.696951 25177 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:21.697923 23928 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:21.695201 23680 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.698407 25332 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.699390 25335 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.709120 25335 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.713614 23925 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.726128 25177 raft_consensus.cc:2955] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.727486 23928 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.734108 23781 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.735972 25176 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:21.738871 25153 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:21.740790 25176 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.741330 23932 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.743228 25344 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.745333 25346 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:21.755848 25337 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.762797 23781 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:21.766916 23681 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.766734 25176 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.772099 25338 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:21.777144 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc9ef29919aa42aaa2b7545c10e39eff with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.780377 25280 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.796140 25334 raft_consensus.cc:1064] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:21.798724 25334 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:21.796204 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 526cc198da134bb590358ed73c23d59e with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:21.802552 23681 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.813212 23781 raft_consensus.cc:2955] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.809959 25176 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.822036 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet cc9ef29919aa42aaa2b7545c10e39eff with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:21.834386 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bec362a6d01840d39be46f0c3d28c218 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.842907 23680 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.845144 23928 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:21.846258 25359 tablet_replica.cc:333] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:21.846551 25176 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:21.847002 25357 tablet_replica.cc:333] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:21.857020 25359 raft_consensus.cc:2243] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.857963 25359 raft_consensus.cc:2272] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.863895 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet bec362a6d01840d39be46f0c3d28c218 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:21.846871 25334 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.864934 25357 raft_consensus.cc:2243] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.865638 25357 raft_consensus.cc:2272] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.866883 25333 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:21.868960 25357 ts_tablet_manager.cc:1916] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.861475 23928 raft_consensus.cc:2955] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.882493 25359 ts_tablet_manager.cc:1916] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.910925 23776 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:21.920622 25357 ts_tablet_manager.cc:1929] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:21.922853 25261 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.925570 25357 log.cc:1199] T 526cc198da134bb590358ed73c23d59e P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/526cc198da134bb590358ed73c23d59e
I20260709 12:13:21.929050 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 526cc198da134bb590358ed73c23d59e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.950525 25334 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.956665 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 526cc198da134bb590358ed73c23d59e (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:21.959014 25176 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.974627 25359 ts_tablet_manager.cc:1929] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:13:21.982126 25359 log.cc:1199] T cc9ef29919aa42aaa2b7545c10e39eff P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/cc9ef29919aa42aaa2b7545c10e39eff
I20260709 12:13:21.980463 23682 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.973680 23928 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:21.997433 25359 tablet_replica.cc:333] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:21.997370 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet cc9ef29919aa42aaa2b7545c10e39eff (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:21.998565 25359 raft_consensus.cc:2243] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.999254 25359 raft_consensus.cc:2272] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:22.015444 25359 ts_tablet_manager.cc:1916] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:22.017154 23777 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:22.054594 23857 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:22.067691 25176 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:22.069406 25362 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:22.074371 23781 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:22.105485 25335 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:22.108207 25334 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:22.113348 25176 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:22.129032 23680 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.136387 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 24dc0370e122479081184511cb1224f1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:22.162163 23784 raft_consensus.cc:2955] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:13:22.203001 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65645.8us
I20260709 12:13:22.203748 25359 ts_tablet_manager.cc:1929] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:22.204241 25359 log.cc:1199] T bec362a6d01840d39be46f0c3d28c218 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/bec362a6d01840d39be46f0c3d28c218
W20260709 12:13:22.204674 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76404.7us
I20260709 12:13:22.210026 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet bec362a6d01840d39be46f0c3d28c218 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:22.212924 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 24dc0370e122479081184511cb1224f1 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:22.221938 25367 tablet_replica.cc:333] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:22.222966 25367 raft_consensus.cc:2243] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:22.224577 25367 raft_consensus.cc:2272] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:22.228163 25367 ts_tablet_manager.cc:1916] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:22.242744 25367 ts_tablet_manager.cc:1929] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:13:22.243279 25367 log.cc:1199] T 24dc0370e122479081184511cb1224f1 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/24dc0370e122479081184511cb1224f1
I20260709 12:13:22.245622 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 24dc0370e122479081184511cb1224f1 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:23.361645 23925 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:23.370314 23856 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:23.370360 23898 consensus_peers.cc:597] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a1b618b31b2e427b8c073b2c24771903. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.370847 23784 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:23.372164 25372 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:23.373765 25370 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:23.384445 25364 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.385927 23780 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.396574 23856 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.402920 23681 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.420675 23925 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:23.450816 23857 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:13:23.451718 23898 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c9d6097cb8d40efa55ff97de5b26130. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.453012 25370 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:23.453585 23784 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:23.458171 25366 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:23.487733 25370 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.501662 23784 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.511703 23857 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.517534 23926 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:23.518366 23682 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:23.547406 23857 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:23.548434 23898 consensus_peers.cc:597] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10c8a875cf4b4acea62c1086994499e4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.548893 25372 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:23.558315 23784 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:23.559811 25372 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:23.564291 23857 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.561808 25372 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.583412 23682 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.605772 23856 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:23.642618 23784 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.664557 23784 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:23.665082 23926 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:23.665459 23824 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 102493ff11494b009f79963bb57be511. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.666399 25369 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:23.667202 25368 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:23.677103 25369 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.679951 23781 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.685942 23925 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:23.698544 23682 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:23.905431 25389 ts_tablet_manager.cc:933] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:23.920855 25389 tablet_copy_client.cc:323] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:23.922688 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 0c9d6097cb8d40efa55ff97de5b26130 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:23.923255 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 0c9d6097cb8d40efa55ff97de5b26130 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-0c9d6097cb8d40efa55ff97de5b26130
I20260709 12:13:23.932646 23944 tablet_copy_source_session.cc:215] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:23.938140 25389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9d6097cb8d40efa55ff97de5b26130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:23.959153 25389 tablet_copy_client.cc:806] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:23.959877 25389 tablet_copy_client.cc:670] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:23.970455 25389 tablet_copy_client.cc:538] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:23.984630 25392 ts_tablet_manager.cc:933] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:23.990252 25389 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:23.992691 25390 ts_tablet_manager.cc:933] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:24.004730 25392 tablet_copy_client.cc:323] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:24.013522 25390 tablet_copy_client.cc:323] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:24.014930 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 10c8a875cf4b4acea62c1086994499e4 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:24.015518 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 10c8a875cf4b4acea62c1086994499e4 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-10c8a875cf4b4acea62c1086994499e4
I20260709 12:13:24.020243 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet a1b618b31b2e427b8c073b2c24771903 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:24.020845 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet a1b618b31b2e427b8c073b2c24771903 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-a1b618b31b2e427b8c073b2c24771903
I20260709 12:13:24.031625 23944 tablet_copy_source_session.cc:215] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:24.035838 23943 tablet_copy_source_session.cc:215] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:24.042841 25390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b618b31b2e427b8c073b2c24771903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.051993 25392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8a875cf4b4acea62c1086994499e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.085229 25389 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.086416 25389 tablet_bootstrap.cc:492] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:24.087419 25389 ts_tablet_manager.cc:1403] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.097s	user 0.022s	sys 0.014s
I20260709 12:13:24.090816 25389 raft_consensus.cc:359] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.091801 25389 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:24.092536 25389 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.103826 25389 ts_tablet_manager.cc:1434] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:13:24.105767 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-0c9d6097cb8d40efa55ff97de5b26130 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:24.106185 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-0c9d6097cb8d40efa55ff97de5b26130 on tablet 0c9d6097cb8d40efa55ff97de5b26130 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:24.112417 25389 ts_tablet_manager.cc:933] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:24.124405 25389 tablet_copy_client.cc:323] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:24.126156 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 102493ff11494b009f79963bb57be511 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:24.126730 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 102493ff11494b009f79963bb57be511 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-102493ff11494b009f79963bb57be511
I20260709 12:13:24.156364 23870 tablet_copy_source_session.cc:215] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:24.163805 25389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102493ff11494b009f79963bb57be511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.201515 25389 tablet_copy_client.cc:806] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.202235 25389 tablet_copy_client.cc:670] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:24.208323 25389 tablet_copy_client.cc:538] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:24.218842 25389 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:24.247771 25390 tablet_copy_client.cc:806] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.260311 25176 raft_consensus.cc:1217] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:13:24.261761 25392 tablet_copy_client.cc:806] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.262342 25392 tablet_copy_client.cc:670] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:24.257252 25389 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.276213 25389 tablet_bootstrap.cc:492] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:24.278489 25389 ts_tablet_manager.cc:1403] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.060s	user 0.025s	sys 0.014s
I20260709 12:13:24.260491 25390 tablet_copy_client.cc:670] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:24.281311 25392 tablet_copy_client.cc:538] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:24.284648 25390 tablet_copy_client.cc:538] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:24.294049 25389 raft_consensus.cc:359] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.303774 25389 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:24.304603 25389 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.310734 25389 ts_tablet_manager.cc:1434] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 12:13:24.301931 25390 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:24.315614 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-102493ff11494b009f79963bb57be511 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:24.316087 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-102493ff11494b009f79963bb57be511 on tablet 102493ff11494b009f79963bb57be511 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:24.313548 25392 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
W20260709 12:13:24.423815 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 55004.8us
I20260709 12:13:24.437321 25392 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.440001 25392 tablet_bootstrap.cc:492] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:24.441039 25392 ts_tablet_manager.cc:1403] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.128s	user 0.021s	sys 0.030s
I20260709 12:13:24.445562 25392 raft_consensus.cc:359] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.446583 25392 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:24.447358 25392 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.451314 25392 ts_tablet_manager.cc:1434] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:13:24.453559 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-10c8a875cf4b4acea62c1086994499e4 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:24.454039 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-10c8a875cf4b4acea62c1086994499e4 on tablet 10c8a875cf4b4acea62c1086994499e4 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:24.481968 25390 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.483040 25390 tablet_bootstrap.cc:492] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:24.483894 25390 ts_tablet_manager.cc:1403] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.182s	user 0.030s	sys 0.022s
I20260709 12:13:24.486966 25390 raft_consensus.cc:359] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.487923 25390 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:24.488580 25390 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:24.491299 25390 ts_tablet_manager.cc:1434] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:13:24.496244 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-a1b618b31b2e427b8c073b2c24771903 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:24.496706 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-a1b618b31b2e427b8c073b2c24771903 on tablet a1b618b31b2e427b8c073b2c24771903 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:24.541112 25176 raft_consensus.cc:1217] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:24.752724 25365 raft_consensus.cc:1064] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:24.759148 25365 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:24.778276 25176 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:24.780288 25366 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:24.792344 23856 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:24.794065 23777 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:24.802161 25366 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:24.810591 25364 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:24.845819 25370 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:24.851312 23780 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:24.853912 23856 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:24.866571 25176 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:24.883546 23681 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:24.943686 25176 raft_consensus.cc:1217] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:24.990633 23925 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:24.994776 25176 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:24.995409 23780 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:24.996119 25365 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:24.997115 25372 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:25.005054 25366 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.008903 25176 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.016763 23780 raft_consensus.cc:2955] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.032092 25177 raft_consensus.cc:1217] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:25.032706 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c9d6097cb8d40efa55ff97de5b26130 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.038640 23683 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.048265 25368 raft_consensus.cc:1064] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:25.053395 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c9d6097cb8d40efa55ff97de5b26130 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:25.059749 25368 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:25.086233 25414 tablet_replica.cc:333] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:25.091310 25176 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.091852 23784 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.093415 23926 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.093335 25403 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:13:25.097203 25414 raft_consensus.cc:2243] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.098102 25414 raft_consensus.cc:2272] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.102552 25414 ts_tablet_manager.cc:1916] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.105424 25331 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.108692 25362 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.132884 25414 ts_tablet_manager.cc:1929] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:13:25.133638 25414 log.cc:1199] T 0c9d6097cb8d40efa55ff97de5b26130 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/0c9d6097cb8d40efa55ff97de5b26130
I20260709 12:13:25.136427 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 0c9d6097cb8d40efa55ff97de5b26130 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:25.139266 25368 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.144821 23926 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.154927 23784 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.143879 25176 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.171286 23683 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.192644 23856 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:25.203532 25176 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.204927 23784 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.205132 25368 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:25.206353 25331 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.226940 25362 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.239712 25177 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.250895 23681 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.253539 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 102493ff11494b009f79963bb57be511 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.256836 23784 raft_consensus.cc:2955] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.265516 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 102493ff11494b009f79963bb57be511 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:25.280278 25423 tablet_replica.cc:333] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:25.282691 25423 raft_consensus.cc:2243] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.284659 25423 raft_consensus.cc:2272] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.289649 25423 ts_tablet_manager.cc:1916] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.302502 25423 ts_tablet_manager.cc:1929] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:25.302872 25423 log.cc:1199] T 102493ff11494b009f79963bb57be511 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/102493ff11494b009f79963bb57be511
I20260709 12:13:25.304535 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 102493ff11494b009f79963bb57be511 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:25.425428 25366 raft_consensus.cc:1064] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:25.428223 25366 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:25.436923 25177 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.438709 23854 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.439009 23784 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.438400 25366 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:13:25.443706 25366 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.445804 25364 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.456643 25364 raft_consensus.cc:1064] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:25.460162 25364 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:25.473377 25176 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.474978 25365 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.476897 23854 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.479727 23777 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:25.479667 25365 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.481304 25370 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.482584 25176 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.486085 25372 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.489781 23777 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.516934 23681 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.527971 23856 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.530700 25366 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.534644 23784 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.549736 23926 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:25.550664 25364 consensus_queue.cc:1175] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "438159d8de3c4027bdcf57eb81dedf86" responder_term: 3 status { last_received { term: 3 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 3 index: 4 } }
I20260709 12:13:25.558063 25176 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.563091 23854 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.574030 25177 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.573585 23683 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.575731 25366 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.579150 23856 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.581017 25370 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.586323 25370 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.597949 25177 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.600265 23854 raft_consensus.cc:2955] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.605787 23925 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:25.613097 23683 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.616832 23856 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.618547 25364 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:13:25.622478 25176 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:25.624321 25365 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.630518 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet a1b618b31b2e427b8c073b2c24771903 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:25.651363 23856 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.643229 25364 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.668530 25433 tablet_replica.cc:333] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:25.669772 25433 raft_consensus.cc:2243] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.670622 25433 raft_consensus.cc:2272] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.681959 25177 raft_consensus.cc:2955] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:25.688287 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 10c8a875cf4b4acea62c1086994499e4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.695322 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a1b618b31b2e427b8c073b2c24771903 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.697162 23683 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.703243 25433 ts_tablet_manager.cc:1916] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.718483 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 10c8a875cf4b4acea62c1086994499e4 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:25.727768 25433 ts_tablet_manager.cc:1929] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:25.728221 25433 log.cc:1199] T a1b618b31b2e427b8c073b2c24771903 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/a1b618b31b2e427b8c073b2c24771903
I20260709 12:13:25.729591 25433 tablet_replica.cc:333] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:25.729607 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet a1b618b31b2e427b8c073b2c24771903 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:25.730528 25433 raft_consensus.cc:2243] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.731176 25433 raft_consensus.cc:2272] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.733917 25433 ts_tablet_manager.cc:1916] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.747582 25433 ts_tablet_manager.cc:1929] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:25.747943 25433 log.cc:1199] T 10c8a875cf4b4acea62c1086994499e4 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/10c8a875cf4b4acea62c1086994499e4
I20260709 12:13:25.749394 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 10c8a875cf4b4acea62c1086994499e4 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:26.785629 23854 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:26.792765 23824 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 643746a7a11243df94746cf1081fc078. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.794612 23925 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:26.795032 23777 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:26.795953 25420 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.796800 25331 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.805683 25438 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.807010 23777 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.806885 23925 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.826555 23926 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:26.830852 23683 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.838418 23781 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:26.839910 25370 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.841084 23856 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:26.842170 23898 consensus_peers.cc:597] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1567865ea02a4884996ae9d8beb9fa29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.843672 25370 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.854496 25432 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.857333 23781 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.875635 23854 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.882441 23925 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:26.894136 23898 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef461146a2924dba8b8e2906d76859f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.894789 23856 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:26.897934 25432 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:26.903888 23781 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:26.905439 25432 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:26.903982 23682 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.936599 25447 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.938299 23854 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.938871 23784 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.955869 23682 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.957403 23854 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:26.965725 23784 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:26.966969 23925 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:26.969589 23824 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f333e0a484248ce88f0905e18500838. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.973255 25439 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.975145 25420 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:26.987296 25438 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:26.987579 23781 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:27.007958 23683 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.012483 23925 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:27.203914 25456 ts_tablet_manager.cc:933] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:27.211099 25456 tablet_copy_client.cc:323] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:27.213791 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 643746a7a11243df94746cf1081fc078 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:27.214371 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 643746a7a11243df94746cf1081fc078 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-643746a7a11243df94746cf1081fc078
I20260709 12:13:27.220429 23870 tablet_copy_source_session.cc:215] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.223644 25456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643746a7a11243df94746cf1081fc078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.244159 25456 tablet_copy_client.cc:806] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.244870 25456 tablet_copy_client.cc:670] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.249450 25456 tablet_copy_client.cc:538] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.259413 25456 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:27.293038 25456 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:27.294083 25456 tablet_bootstrap.cc:492] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:27.294905 25456 ts_tablet_manager.cc:1403] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.021s
I20260709 12:13:27.298197 25456 raft_consensus.cc:359] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.299057 25456 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:27.299787 25456 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.308007 25456 ts_tablet_manager.cc:1434] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 12:13:27.309726 25460 ts_tablet_manager.cc:933] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:27.318608 25460 tablet_copy_client.cc:323] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:27.324322 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 1567865ea02a4884996ae9d8beb9fa29 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:27.324973 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 1567865ea02a4884996ae9d8beb9fa29 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-1567865ea02a4884996ae9d8beb9fa29
I20260709 12:13:27.330937 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-643746a7a11243df94746cf1081fc078 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:27.331442 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-643746a7a11243df94746cf1081fc078 on tablet 643746a7a11243df94746cf1081fc078 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:27.334339 23943 tablet_copy_source_session.cc:215] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.337738 25460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1567865ea02a4884996ae9d8beb9fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.358497 25460 tablet_copy_client.cc:806] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.359210 25460 tablet_copy_client.cc:670] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.363839 25460 tablet_copy_client.cc:538] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.366720 25456 ts_tablet_manager.cc:933] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:27.369396 25456 tablet_copy_client.cc:323] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:27.371469 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet ef461146a2924dba8b8e2906d76859f1 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:27.371969 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet ef461146a2924dba8b8e2906d76859f1 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-ef461146a2924dba8b8e2906d76859f1
I20260709 12:13:27.377337 23943 tablet_copy_source_session.cc:215] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.379463 25460 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:27.381129 25456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef461146a2924dba8b8e2906d76859f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.397724 25456 tablet_copy_client.cc:806] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.398259 25456 tablet_copy_client.cc:670] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.404223 25456 tablet_copy_client.cc:538] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.410622 25460 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:27.411767 25460 tablet_bootstrap.cc:492] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:27.412559 25460 ts_tablet_manager.cc:1403] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.021s
I20260709 12:13:27.415910 25460 raft_consensus.cc:359] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.417152 25460 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:27.418354 25456 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:27.417894 25460 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.420904 25460 ts_tablet_manager.cc:1434] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 12:13:27.422739 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-1567865ea02a4884996ae9d8beb9fa29 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:27.423177 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-1567865ea02a4884996ae9d8beb9fa29 on tablet 1567865ea02a4884996ae9d8beb9fa29 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:27.431620 25460 ts_tablet_manager.cc:933] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:27.440828 25460 tablet_copy_client.cc:323] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:27.445333 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 8f333e0a484248ce88f0905e18500838 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:27.446012 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 8f333e0a484248ce88f0905e18500838 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-8f333e0a484248ce88f0905e18500838
I20260709 12:13:27.458520 25456 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:27.459550 25456 tablet_bootstrap.cc:492] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:27.460270 25456 ts_tablet_manager.cc:1403] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.016s
I20260709 12:13:27.460887 23870 tablet_copy_source_session.cc:215] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.463234 25456 raft_consensus.cc:359] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.464180 25460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f333e0a484248ce88f0905e18500838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.464130 25456 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:27.465328 25456 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.467098 25456 ts_tablet_manager.cc:1434] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:13:27.468920 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-ef461146a2924dba8b8e2906d76859f1 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:27.469345 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-ef461146a2924dba8b8e2906d76859f1 on tablet ef461146a2924dba8b8e2906d76859f1 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:27.484298 25460 tablet_copy_client.cc:806] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.484884 25460 tablet_copy_client.cc:670] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.490424 25460 tablet_copy_client.cc:538] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.500699 25460 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:27.528911 25460 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:27.529690 25460 tablet_bootstrap.cc:492] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:27.530224 25460 ts_tablet_manager.cc:1403] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.008s
I20260709 12:13:27.532491 25460 raft_consensus.cc:359] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.533113 25460 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:27.533820 25460 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:27.535822 25460 ts_tablet_manager.cc:1434] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:13:27.537627 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8f333e0a484248ce88f0905e18500838 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:27.537973 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8f333e0a484248ce88f0905e18500838 on tablet 8f333e0a484248ce88f0905e18500838 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:27.584265 25176 raft_consensus.cc:1217] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:27.879704 25176 raft_consensus.cc:1217] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:27.919617 25176 raft_consensus.cc:1217] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:27.961642 25176 raft_consensus.cc:1217] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 12:13:27.973058 25439 raft_consensus.cc:1064] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:27.976336 25439 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:27.982690 23925 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:27.982682 25176 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:27.984813 23781 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:27.985319 25468 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:27.986402 25439 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:27.987401 25420 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.002333 25439 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.021520 25176 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.023309 23926 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.029654 23683 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:28.034818 23784 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.096015 23854 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.101759 25176 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.103027 23784 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.103494 25420 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.104605 25470 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.113669 25420 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.127271 25176 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.133847 23777 raft_consensus.cc:2955] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.150589 23682 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.154114 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 643746a7a11243df94746cf1081fc078 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.168746 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 643746a7a11243df94746cf1081fc078 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:28.170837 25481 tablet_replica.cc:333] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:28.171947 25481 raft_consensus.cc:2243] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:28.173084 25481 raft_consensus.cc:2272] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:28.177263 25481 ts_tablet_manager.cc:1916] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:28.194373 25481 ts_tablet_manager.cc:1929] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:28.194761 25481 log.cc:1199] T 643746a7a11243df94746cf1081fc078 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/643746a7a11243df94746cf1081fc078
I20260709 12:13:28.196241 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 643746a7a11243df94746cf1081fc078 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:28.314198 25447 raft_consensus.cc:1064] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:28.316934 25447 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.324165 25176 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:28.325594 23856 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:28.325724 25472 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.327749 23781 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:28.329231 25447 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.331038 25432 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.342600 25447 raft_consensus.cc:1064] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:28.345721 25447 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.411000 25448 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.412125 23784 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.432533 23683 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.433597 23856 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.453917 25469 raft_consensus.cc:1064] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:28.455732 23856 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.456497 25469 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 1, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.457316 25432 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.460862 25176 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.462385 23784 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.465240 25432 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.467072 25432 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.466467 25176 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.494786 25177 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:28.495843 23926 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:28.497571 23781 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:28.498899 25439 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.500490 25438 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.503216 25420 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.517047 23925 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.525444 25176 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.525951 23854 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.527443 25472 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.527963 25432 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.529398 25447 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.532446 25420 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.536175 23780 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.535250 23926 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.537540 25472 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.552579 25174 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.559239 25175 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.561661 23683 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.558156 23781 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.543352 25176 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.569238 23681 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:13:28.584878 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57541us
I20260709 12:13:28.590090 23682 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.597735 23854 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.601496 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1567865ea02a4884996ae9d8beb9fa29 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.604040 25176 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:28.604523 23854 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.605690 25469 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.606154 23856 raft_consensus.cc:2955] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.615475 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 1567865ea02a4884996ae9d8beb9fa29 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:28.616835 23780 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:28.619220 25420 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.624120 25438 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.631116 25174 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.637629 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f333e0a484248ce88f0905e18500838 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.639520 25498 tablet_replica.cc:333] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:28.656085 23780 raft_consensus.cc:2955] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.674502 23679 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.666162 25498 raft_consensus.cc:2243] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:28.680680 25498 raft_consensus.cc:2272] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:28.692982 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f333e0a484248ce88f0905e18500838 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:28.694056 25481 tablet_replica.cc:333] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:28.695101 25481 raft_consensus.cc:2243] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:28.695179 25498 ts_tablet_manager.cc:1916] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:28.699010 25481 raft_consensus.cc:2272] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:28.702710 25481 ts_tablet_manager.cc:1916] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:28.703022 23926 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:28.712229 25174 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:28.713526 23784 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:28.713882 25497 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:28.716897 25497 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.725104 25497 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.732204 25174 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.732204 23784 raft_consensus.cc:2955] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.735628 25481 ts_tablet_manager.cc:1929] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.4
I20260709 12:13:28.736155 25481 log.cc:1199] T 8f333e0a484248ce88f0905e18500838 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/8f333e0a484248ce88f0905e18500838
I20260709 12:13:28.746495 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 8f333e0a484248ce88f0905e18500838 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:28.752130 23683 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 1 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:28.764626 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ef461146a2924dba8b8e2906d76859f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.774708 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet ef461146a2924dba8b8e2906d76859f1 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:28.776408 25498 ts_tablet_manager.cc:1929] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:28.776710 25499 tablet_replica.cc:333] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:28.776898 25498 log.cc:1199] T 1567865ea02a4884996ae9d8beb9fa29 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/1567865ea02a4884996ae9d8beb9fa29
I20260709 12:13:28.777529 25499 raft_consensus.cc:2243] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:28.778216 25499 raft_consensus.cc:2272] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:28.778360 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 1567865ea02a4884996ae9d8beb9fa29 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:28.780807 25499 ts_tablet_manager.cc:1916] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:28.793952 25499 ts_tablet_manager.cc:1929] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:28.794447 25499 log.cc:1199] T ef461146a2924dba8b8e2906d76859f1 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/ef461146a2924dba8b8e2906d76859f1
I20260709 12:13:28.795924 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet ef461146a2924dba8b8e2906d76859f1 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:29.844846 23926 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:29.850226 23856 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:29.850502 23784 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:13:29.850638 23898 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f44bb7910714c46af4e3d0e2076e7c5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:29.851528 25502 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:29.852376 25488 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:29.861203 25502 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.863006 23784 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.871788 23854 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.881076 23682 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.904758 23854 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:29.911731 23926 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:29.916162 23784 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:29.917654 25469 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:29.919396 25470 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:29.919453 23824 consensus_peers.cc:597] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9f2f570298740c691fa19d449b3fa4f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:29.929311 25469 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.932240 23926 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.932240 23784 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.947235 23780 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:29.955284 23925 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:29.957430 23749 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 349b687db5c44196bafc207c26ca3679. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:29.958242 25508 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:29.958771 23854 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:29.951201 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.961268 25500 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:29.992183 25508 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.993856 23856 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:29.997532 23928 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.012192 23683 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 2, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.016530 23925 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.039142 23856 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:13:30.039980 23898 consensus_peers.cc:597] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 233089f4b82a4b1996ad9129dc392819. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:30.040488 23780 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:30.041185 25502 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.042333 25501 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.056970 25488 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.058609 23856 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.067864 23784 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.077211 23679 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:30.246316 25523 ts_tablet_manager.cc:933] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:30.248672 25523 tablet_copy_client.cc:323] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:30.250248 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 8f44bb7910714c46af4e3d0e2076e7c5 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:30.250705 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 8f44bb7910714c46af4e3d0e2076e7c5 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-8f44bb7910714c46af4e3d0e2076e7c5
I20260709 12:13:30.257067 23943 tablet_copy_source_session.cc:215] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:30.260298 25523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f44bb7910714c46af4e3d0e2076e7c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:30.274634 25523 tablet_copy_client.cc:806] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:30.275421 25523 tablet_copy_client.cc:670] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:30.280115 25523 tablet_copy_client.cc:538] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:30.293888 25523 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:30.319573 25523 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:30.320590 25523 tablet_bootstrap.cc:492] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:30.321372 25523 ts_tablet_manager.cc:1403] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.007s
I20260709 12:13:30.324399 25523 raft_consensus.cc:359] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.325259 25523 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:30.325906 25523 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.329447 25523 ts_tablet_manager.cc:1434] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 12:13:30.331156 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8f44bb7910714c46af4e3d0e2076e7c5 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:30.331627 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-8f44bb7910714c46af4e3d0e2076e7c5 on tablet 8f44bb7910714c46af4e3d0e2076e7c5 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:30.402192 25523 ts_tablet_manager.cc:933] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:30.404031 25523 tablet_copy_client.cc:323] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:30.405452 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet d9f2f570298740c691fa19d449b3fa4f from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:30.405884 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet d9f2f570298740c691fa19d449b3fa4f from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-d9f2f570298740c691fa19d449b3fa4f
I20260709 12:13:30.411499 23870 tablet_copy_source_session.cc:215] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:30.415045 25523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9f2f570298740c691fa19d449b3fa4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:30.427150 25523 tablet_copy_client.cc:806] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:30.427716 25523 tablet_copy_client.cc:670] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:30.430958 25523 tablet_copy_client.cc:538] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:30.441309 25523 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:30.441402 25528 ts_tablet_manager.cc:933] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:30.452858 25528 tablet_copy_client.cc:323] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:13:30.454965 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet 349b687db5c44196bafc207c26ca3679 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:13:30.455521 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet 349b687db5c44196bafc207c26ca3679 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-349b687db5c44196bafc207c26ca3679
I20260709 12:13:30.470709 25523 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:30.471694 25523 tablet_bootstrap.cc:492] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:30.472440 25523 ts_tablet_manager.cc:1403] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.007s
I20260709 12:13:30.471616 23795 tablet_copy_source_session.cc:215] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:30.475478 25523 raft_consensus.cc:359] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.476236 25528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 349b687db5c44196bafc207c26ca3679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:30.476322 25523 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:30.477137 25523 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.479882 25523 ts_tablet_manager.cc:1434] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:13:30.481945 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d9f2f570298740c691fa19d449b3fa4f received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:30.483208 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d9f2f570298740c691fa19d449b3fa4f on tablet d9f2f570298740c691fa19d449b3fa4f with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:30.499101 25528 tablet_copy_client.cc:806] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:30.499750 25528 tablet_copy_client.cc:670] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:30.522507 25528 tablet_copy_client.cc:538] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:30.532933 25528 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:30.562134 25528 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:30.563030 25528 tablet_bootstrap.cc:492] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:30.563741 25528 ts_tablet_manager.cc:1403] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.008s
I20260709 12:13:30.563850 25523 ts_tablet_manager.cc:933] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:30.565680 25523 tablet_copy_client.cc:323] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:30.567111 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 233089f4b82a4b1996ad9129dc392819 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:30.566826 25528 raft_consensus.cc:359] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.567544 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 233089f4b82a4b1996ad9129dc392819 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-233089f4b82a4b1996ad9129dc392819
I20260709 12:13:30.567751 25528 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:30.568461 25528 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.570892 25528 ts_tablet_manager.cc:1434] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:13:30.573176 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-349b687db5c44196bafc207c26ca3679 received from {username='slave'} at 127.0.0.1:40134
I20260709 12:13:30.573617 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-349b687db5c44196bafc207c26ca3679 on tablet 349b687db5c44196bafc207c26ca3679 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:30.576086 23943 tablet_copy_source_session.cc:215] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:30.580744 25523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233089f4b82a4b1996ad9129dc392819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:30.596783 25523 tablet_copy_client.cc:806] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:30.597321 25523 tablet_copy_client.cc:670] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:30.600517 25523 tablet_copy_client.cc:538] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:30.609357 25523 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:30.630095 25523 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:30.630975 25523 tablet_bootstrap.cc:492] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:30.631649 25523 ts_tablet_manager.cc:1403] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:13:30.634222 25523 raft_consensus.cc:359] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.635004 25523 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:30.635623 25523 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:30.637756 25523 ts_tablet_manager.cc:1434] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:13:30.649005 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-233089f4b82a4b1996ad9129dc392819 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:30.649462 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-233089f4b82a4b1996ad9129dc392819 on tablet 233089f4b82a4b1996ad9129dc392819 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:30.760039 25174 raft_consensus.cc:1217] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:13:30.982074 25174 raft_consensus.cc:1217] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:30.995368 25174 raft_consensus.cc:1217] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:31.111202 25174 raft_consensus.cc:1217] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:31.359488 25488 raft_consensus.cc:1064] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:31.362011 25488 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.371619 25174 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:31.373191 25497 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.373466 23856 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:31.373888 23781 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:31.377185 25538 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.380729 25497 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.395385 25538 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.397239 23781 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.400339 23854 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.416394 23683 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.419816 25174 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.432488 25537 raft_consensus.cc:1064] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:31.439751 25537 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.465058 25174 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:31.466266 23928 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:31.466781 25537 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:31.468148 23853 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:31.469377 25536 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:31.470544 25508 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:31.492718 23925 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.500142 25176 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:31.506703 25488 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:31.510619 25508 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.519294 25531 raft_consensus.cc:1064] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:31.522042 25531 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.525590 23780 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:31.527150 23925 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.527731 23854 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.537122 25522 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.539235 25497 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:31.536201 25176 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.514164 25174 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.548620 23679 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 2 to 3, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:31.570788 23682 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.584878 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f44bb7910714c46af4e3d0e2076e7c5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:31.585850 23780 raft_consensus.cc:2955] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.596621 23784 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:31.599896 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f44bb7910714c46af4e3d0e2076e7c5 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:31.601048 25531 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.612314 23781 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.618835 25174 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:31.618971 23925 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:31.621342 25537 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.623307 25537 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:31.629127 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 93357us
I20260709 12:13:31.637269 25556 tablet_replica.cc:333] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:31.653007 25175 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:31.656562 25522 raft_consensus.cc:1064] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:31.656100 25508 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.657720 23925 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.660050 25522 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.664317 25174 raft_consensus.cc:2955] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.674328 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 349b687db5c44196bafc207c26ca3679 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:31.679534 23928 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:31.682355 23681 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 1 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:13:31.694007 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51295.5us
I20260709 12:13:31.696748 25175 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:31.698323 25552 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.701640 23854 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:31.704064 25497 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.708894 25551 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.715111 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 349b687db5c44196bafc207c26ca3679 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:31.717491 25556 raft_consensus.cc:2243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:31.720136 25556 raft_consensus.cc:2272] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:31.724184 25556 ts_tablet_manager.cc:1916] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:31.738252 23784 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:31.740955 25522 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:31.744369 25555 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:31.759840 25497 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.772755 23853 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.796758 23776 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.791247 25175 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.797803 25556 ts_tablet_manager.cc:1929] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:13:31.800431 25556 log.cc:1199] T 8f44bb7910714c46af4e3d0e2076e7c5 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/8f44bb7910714c46af4e3d0e2076e7c5
I20260709 12:13:31.802474 25556 tablet_replica.cc:333] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:31.803458 25556 raft_consensus.cc:2243] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:31.804193 25556 raft_consensus.cc:2272] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:31.806197 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 8f44bb7910714c46af4e3d0e2076e7c5 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:31.816408 25556 ts_tablet_manager.cc:1916] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:31.817399 25558 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.825196 23679 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:31.840454 25556 ts_tablet_manager.cc:1929] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:31.841012 25556 log.cc:1199] T 349b687db5c44196bafc207c26ca3679 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/349b687db5c44196bafc207c26ca3679
I20260709 12:13:31.842782 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 349b687db5c44196bafc207c26ca3679 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:31.844007 25174 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.846441 23925 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.853770 23928 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.860291 25175 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:31.861779 25552 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.865221 23853 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:31.867076 23784 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.868005 25522 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.869428 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.873432 25554 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.880744 25174 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.884209 23857 raft_consensus.cc:2955] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.893671 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 233089f4b82a4b1996ad9129dc392819 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:31.920660 23680 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:31.925135 23857 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:31.931344 25174 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:31.933043 25551 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.939631 23925 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:31.941929 25558 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:31.944483 25531 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.950654 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 233089f4b82a4b1996ad9129dc392819 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:31.950924 25175 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.961690 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d9f2f570298740c691fa19d449b3fa4f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:31.967494 23925 raft_consensus.cc:2955] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:31.989205 23680 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:32.000770 25566 tablet_replica.cc:333] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:32.002205 25566 raft_consensus.cc:2243] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:32.003031 25566 raft_consensus.cc:2272] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:32.005775 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet d9f2f570298740c691fa19d449b3fa4f with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:32.006539 25566 ts_tablet_manager.cc:1916] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:32.019903 25566 ts_tablet_manager.cc:1929] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:32.020298 25566 log.cc:1199] T 233089f4b82a4b1996ad9129dc392819 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/233089f4b82a4b1996ad9129dc392819
I20260709 12:13:32.021807 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 233089f4b82a4b1996ad9129dc392819 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:32.022033 25566 tablet_replica.cc:333] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:32.023105 25566 raft_consensus.cc:2243] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:32.023859 25566 raft_consensus.cc:2272] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:32.026270 25566 ts_tablet_manager.cc:1916] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:32.039803 25566 ts_tablet_manager.cc:1929] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:32.040195 25566 log.cc:1199] T d9f2f570298740c691fa19d449b3fa4f P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/d9f2f570298740c691fa19d449b3fa4f
I20260709 12:13:32.041612 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet d9f2f570298740c691fa19d449b3fa4f (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:33.078024 23853 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.084628 23928 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:33.084753 23784 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:13:33.084822 23824 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34d6300a13a3401283a46bf3d2a07e82. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:33.086141 25568 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:33.087313 25558 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:33.094363 25551 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.100114 23776 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.114015 23928 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.114676 23680 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:33.141819 23925 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.179771 23853 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:33.189502 23898 consensus_peers.cc:597] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7203e0ca92840388d931c03974d32ff. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:33.179680 23776 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:33.191294 25497 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.193468 25554 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.208808 25554 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.212661 23776 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.226845 23853 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.232492 23681 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.235869 23857 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.259884 23925 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:33.261497 23824 consensus_peers.cc:597] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71cb2ce294574daa8d7244ae252e8aa6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:33.261636 25576 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.263557 23776 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:33.266417 25577 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.278203 25576 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.281178 23784 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.289815 23928 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.293727 23925 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.296267 23682 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:33.320432 23898 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3aacc8d80dd2433b94e6d07927304b47. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:33.320794 23776 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:33.321731 23853 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:33.322953 25554 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.323854 25585 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:33.334228 25497 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.343919 23853 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.360589 23682 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.362967 23784 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:33.562290 25592 ts_tablet_manager.cc:933] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:33.564168 25592 tablet_copy_client.cc:323] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:33.565737 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet c7203e0ca92840388d931c03974d32ff from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:33.566241 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet c7203e0ca92840388d931c03974d32ff from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-c7203e0ca92840388d931c03974d32ff
I20260709 12:13:33.573843 23943 tablet_copy_source_session.cc:215] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:33.576772 25592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7203e0ca92840388d931c03974d32ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:33.592743 25592 tablet_copy_client.cc:806] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:33.593933 25592 tablet_copy_client.cc:670] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:33.603434 25592 tablet_copy_client.cc:538] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:33.612985 25592 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:33.634500 25592 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:33.635218 25592 tablet_bootstrap.cc:492] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:33.635802 25592 ts_tablet_manager.cc:1403] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.007s
I20260709 12:13:33.638392 25592 raft_consensus.cc:359] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.639072 25592 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:33.639726 25592 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.643437 25592 ts_tablet_manager.cc:1434] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:13:33.648829 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c7203e0ca92840388d931c03974d32ff received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:33.649325 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c7203e0ca92840388d931c03974d32ff on tablet c7203e0ca92840388d931c03974d32ff with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:33.665337 25592 ts_tablet_manager.cc:933] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:33.671241 25592 tablet_copy_client.cc:323] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:33.672943 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 34d6300a13a3401283a46bf3d2a07e82 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:33.673488 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 34d6300a13a3401283a46bf3d2a07e82 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-34d6300a13a3401283a46bf3d2a07e82
I20260709 12:13:33.685025 23870 tablet_copy_source_session.cc:215] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:33.690128 25592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d6300a13a3401283a46bf3d2a07e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:33.706614 25596 ts_tablet_manager.cc:933] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:33.706905 25592 tablet_copy_client.cc:806] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:33.707494 25592 tablet_copy_client.cc:670] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:33.708423 25596 tablet_copy_client.cc:323] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:33.710019 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 3aacc8d80dd2433b94e6d07927304b47 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:33.710445 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 3aacc8d80dd2433b94e6d07927304b47 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-3aacc8d80dd2433b94e6d07927304b47
I20260709 12:13:33.711293 25592 tablet_copy_client.cc:538] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:33.721922 23943 tablet_copy_source_session.cc:215] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:33.727691 25596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3aacc8d80dd2433b94e6d07927304b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:33.733372 25592 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:33.750850 25596 tablet_copy_client.cc:806] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:33.751447 25596 tablet_copy_client.cc:670] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:33.755448 25596 tablet_copy_client.cc:538] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:33.765415 25596 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:33.789613 25592 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:33.790581 25592 tablet_bootstrap.cc:492] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:33.791298 25592 ts_tablet_manager.cc:1403] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.058s	user 0.028s	sys 0.001s
I20260709 12:13:33.794401 25592 raft_consensus.cc:359] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.795265 25592 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:33.795975 25592 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.798439 25592 ts_tablet_manager.cc:1434] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:13:33.801030 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-34d6300a13a3401283a46bf3d2a07e82 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:33.801496 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-34d6300a13a3401283a46bf3d2a07e82 on tablet 34d6300a13a3401283a46bf3d2a07e82 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:33.817500 25596 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:33.818485 25596 tablet_bootstrap.cc:492] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:33.819154 25596 ts_tablet_manager.cc:1403] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.007s
I20260709 12:13:33.822085 25596 raft_consensus.cc:359] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.822891 25596 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:33.823526 25596 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.825877 25596 ts_tablet_manager.cc:1434] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.004s
I20260709 12:13:33.832248 23943 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-3aacc8d80dd2433b94e6d07927304b47 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:33.832625 23943 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-3aacc8d80dd2433b94e6d07927304b47 on tablet 3aacc8d80dd2433b94e6d07927304b47 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:33.877216 25596 ts_tablet_manager.cc:933] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:33.878877 25596 tablet_copy_client.cc:323] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:33.880800 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 71cb2ce294574daa8d7244ae252e8aa6 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:33.881366 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 71cb2ce294574daa8d7244ae252e8aa6 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-71cb2ce294574daa8d7244ae252e8aa6
I20260709 12:13:33.889643 23870 tablet_copy_source_session.cc:215] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:33.892822 25596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cb2ce294574daa8d7244ae252e8aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:33.907262 25596 tablet_copy_client.cc:806] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:33.907919 25596 tablet_copy_client.cc:670] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:33.911494 25596 tablet_copy_client.cc:538] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:33.937219 25596 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:33.966763 25596 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:33.967731 25596 tablet_bootstrap.cc:492] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:33.968470 25596 ts_tablet_manager.cc:1403] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.032s	user 0.025s	sys 0.004s
I20260709 12:13:33.971477 25596 raft_consensus.cc:359] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.972466 25596 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:33.973153 25596 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:33.975647 25596 ts_tablet_manager.cc:1434] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:13:33.977412 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-71cb2ce294574daa8d7244ae252e8aa6 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:33.977823 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-71cb2ce294574daa8d7244ae252e8aa6 on tablet 71cb2ce294574daa8d7244ae252e8aa6 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:34.011214 25175 raft_consensus.cc:1217] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:34.056623 25175 raft_consensus.cc:1217] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:13:34.194288 25175 raft_consensus.cc:1217] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:34.415617 25567 raft_consensus.cc:1064] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:34.417980 25567 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:34.430605 25175 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:34.431622 23777 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:34.432268 23857 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:34.432998 25585 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:34.435287 25175 raft_consensus.cc:1217] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:34.437335 25567 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:34.438953 25497 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:34.459403 25605 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.462003 25175 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.462411 23857 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.479282 23777 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.480047 23681 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.519153 23928 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:34.526496 25175 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:34.528191 25567 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:34.529199 23857 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:34.532459 25497 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:34.537792 25605 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.543401 25175 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.549119 23857 raft_consensus.cc:2955] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.566984 23680 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.581430 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c7203e0ca92840388d931c03974d32ff with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:34.585458 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet c7203e0ca92840388d931c03974d32ff with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:34.588047 25615 tablet_replica.cc:333] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:34.591719 25615 raft_consensus.cc:2243] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:34.592458 25615 raft_consensus.cc:2272] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:34.600613 25615 ts_tablet_manager.cc:1916] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:34.604982 25603 raft_consensus.cc:1064] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:34.607649 25603 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:34.614696 25175 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:34.616839 23926 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:34.617090 25615 ts_tablet_manager.cc:1929] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:34.616868 23777 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:34.617525 25615 log.cc:1199] T c7203e0ca92840388d931c03974d32ff P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/c7203e0ca92840388d931c03974d32ff
I20260709 12:13:34.618665 25577 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:34.619112 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet c7203e0ca92840388d931c03974d32ff (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:34.619881 25603 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:34.620817 25551 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:34.654584 25577 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.670786 25175 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.688983 23784 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.695281 23926 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.745675 23683 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:34.916343 25629 raft_consensus.cc:1064] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:34.920833 25629 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:34.930914 25605 raft_consensus.cc:1064] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:34.935078 25605 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:34.947746 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:34.966445 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 155000us
I20260709 12:13:34.986775 23857 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:34.989334 25625 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:35.010469 25175 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:35.013306 25577 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:35.014106 23776 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:35.015379 25625 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:35.016902 23926 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:35.023511 25621 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:35.031250 25175 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:35.041922 23853 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.043390 25605 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:35.058748 25605 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.063412 23784 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:35.063571 25175 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:35.068514 25630 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:35.082090 25174 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.082669 25627 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.084235 23926 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.088333 23779 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.080120 25629 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.122411 23683 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.132910 23680 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.143862 23779 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:35.134025 23682 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.146965 25577 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:35.152940 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 34d6300a13a3401283a46bf3d2a07e82 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:35.168506 25603 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:35.181207 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 53233.9us
I20260709 12:13:35.180895 23779 raft_consensus.cc:2955] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.184239 25174 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.203894 23853 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.246876 25640 tablet_replica.cc:333] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:35.259755 23781 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.252290 25177 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.276827 25640 raft_consensus.cc:2243] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:35.287629 25640 raft_consensus.cc:2272] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:35.300781 25640 ts_tablet_manager.cc:1916] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:35.371804 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34d6300a13a3401283a46bf3d2a07e82 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:35.370770 23928 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.384305 23779 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:35.384922 25175 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:35.386273 25585 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:35.387337 25614 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:35.396443 25585 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.398159 25175 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.416781 25640 ts_tablet_manager.cc:1929] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:13:35.417691 25640 log.cc:1199] T 34d6300a13a3401283a46bf3d2a07e82 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/34d6300a13a3401283a46bf3d2a07e82
I20260709 12:13:35.419833 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 34d6300a13a3401283a46bf3d2a07e82 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:35.421521 23781 raft_consensus.cc:2955] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.429289 23853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:35.430033 23853 raft_consensus.cc:606] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:35.442003 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "526cc198da134bb590358ed73c23d59e"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:35.442644 23856 raft_consensus.cc:606] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:35.449167 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f333e0a484248ce88f0905e18500838"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:35.449783 23856 raft_consensus.cc:606] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:35.452342 23680 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.472633 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3aacc8d80dd2433b94e6d07927304b47 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:35.477201 25630 raft_consensus.cc:993] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:35.477792 25630 raft_consensus.cc:1081] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:35.479537 25175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "526cc198da134bb590358ed73c23d59e"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44500
I20260709 12:13:35.480209 25175 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:35.480563 25175 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:35.483781 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 3aacc8d80dd2433b94e6d07927304b47 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:35.487560 25175 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.491493 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:35.492266 23928 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:35.494660 25645 tablet_replica.cc:333] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:35.496371 25645 raft_consensus.cc:2243] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:35.497812 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:35.495088 25175 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:35.500818 23853 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:35.501178 23853 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:35.501968 23853 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.504072 23928 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:35.505730 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:35.506716 25595 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:35.516158 25645 raft_consensus.cc:2272] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:35.504271 23853 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:35.520254 25645 ts_tablet_manager.cc:1916] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:35.527949 23925 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:35.528685 23925 raft_consensus.cc:606] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:35.533012 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:35.533747 23856 raft_consensus.cc:606] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:35.533119 25595 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:35.533074 23853 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:35.534971 25595 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.539204 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:35.539966 23856 raft_consensus.cc:606] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:35.545519 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:13:35.547245 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:35.548347 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:35.572428 25645 ts_tablet_manager.cc:1929] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:35.570451 23683 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 3 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.572995 25645 log.cc:1199] T 3aacc8d80dd2433b94e6d07927304b47 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/3aacc8d80dd2433b94e6d07927304b47
I20260709 12:13:35.575165 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 3aacc8d80dd2433b94e6d07927304b47 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:35.727015 25627 raft_consensus.cc:993] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:35.727679 25551 raft_consensus.cc:1081] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:35.729458 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:35.730031 23780 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:35.730715 23780 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:35.741533 23780 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.743723 23780 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:35.745102 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:35.745102 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:35.745710 23853 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:35.745939 23853 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:35.745885 23925 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:35.746703 23853 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.747558 23853 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:35.752203 23853 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:35.753644 23751 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:35.754552 25622 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:35.755069 25622 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:35.755926 25622 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.764340 23925 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:35.773188 23683 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 2 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.790081 23858 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.803129 23780 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:35.803447 25175 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:35.804414 25630 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:35.808257 25627 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:35.813783 25627 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.820087 25175 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.830708 23779 raft_consensus.cc:2955] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.834241 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 71cb2ce294574daa8d7244ae252e8aa6 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:35.847263 25629 raft_consensus.cc:993] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:35.847800 25629 raft_consensus.cc:1081] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:35.849270 25175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f333e0a484248ce88f0905e18500838"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44500
I20260709 12:13:35.849912 25175 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:35.850261 25175 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:35.851966 23680 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:35.857023 25175 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.861617 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:35.862365 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:35.862947 23779 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:35.862212 23858 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:35.866165 23858 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:35.866285 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 71cb2ce294574daa8d7244ae252e8aa6 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:35.867225 25640 tablet_replica.cc:333] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:35.867007 23858 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.868317 25640 raft_consensus.cc:2243] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:35.869119 25640 raft_consensus.cc:2272] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:35.868428 23858 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:35.873283 23779 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:35.879794 25114 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:35.861070 25175 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:35.881263 25595 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:35.881767 25595 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:35.882660 25595 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:35.877599 25640 ts_tablet_manager.cc:1916] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:35.895574 25605 raft_consensus.cc:993] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:35.896160 25605 raft_consensus.cc:1081] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:35.899048 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:35.899626 23781 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:35.899981 23781 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:35.901142 23858 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:35.907428 23781 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.909206 23781 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:35.909852 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:35.910184 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:35.909667 23683 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.910444 23853 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:35.910750 23925 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:35.911051 23925 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:35.911676 23925 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.912750 23925 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:35.916635 23853 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:35.918538 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:35.918967 23925 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:35.920965 25623 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:35.921468 25623 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:35.922379 25623 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:35.930305 25640 ts_tablet_manager.cc:1929] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:35.930711 25640 log.cc:1199] T 71cb2ce294574daa8d7244ae252e8aa6 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/71cb2ce294574daa8d7244ae252e8aa6
I20260709 12:13:35.932139 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 71cb2ce294574daa8d7244ae252e8aa6 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:35.940137 23683 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.017822 25627 raft_consensus.cc:993] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.018491 25627 raft_consensus.cc:1081] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.020188 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:36.020711 23779 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:36.021039 23779 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.026150 23779 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.027695 23779 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:36.028353 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:36.028550 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:36.028985 23858 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:36.029145 23925 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.029379 23858 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:36.030306 23858 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.031490 23858 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.035537 23925 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.036741 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:36.037395 25623 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:36.037626 23858 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.037915 25623 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:36.038759 25623 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.049196 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.051245 23858 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:36.052587 25595 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:36.070468 23928 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:36.071682 25646 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:36.147099 25551 raft_consensus.cc:993] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.147514 25627 raft_consensus.cc:1081] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.149051 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:36.149539 23779 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:36.149891 23779 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.155836 23779 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.157200 23779 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:36.158061 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:36.157965 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:36.158597 23925 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.158725 23858 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:36.159055 23858 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:36.159787 23858 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.160610 23858 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.163223 23925 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.164149 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:36.164746 25623 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:36.164872 23858 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.165136 25623 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:36.166035 25623 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.175062 23683 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.266805 23858 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:36.267990 25622 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:36.273396 23925 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:36.275048 25622 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:36.300113 23858 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:36.301267 25595 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:36.304695 23779 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:36.306404 25595 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:36.309037 23925 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:36.310665 25660 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:36.355981 23858 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:36.358616 25660 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:36.475814 23858 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:36.477178 25660 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:36.491575 23925 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:36.494611 25622 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:36.616871 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:36.620985 23925 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:36.622619 25622 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:36.633376 23858 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:36.634856 25622 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:36.676535 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:36.677183 23857 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.683579 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:36.684880 23858 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.687518 23928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:36.688102 23928 raft_consensus.cc:606] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.690464 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:36.690891 23858 raft_consensus.cc:606] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.693017 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:36.693416 23858 raft_consensus.cc:606] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.695700 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:36.696125 23858 raft_consensus.cc:606] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:36.696971 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:13:36.698060 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:36.698622 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:36.761793 25629 raft_consensus.cc:993] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.762281 25627 raft_consensus.cc:1081] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.764917 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:36.765472 23779 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:36.765810 23779 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.772176 23779 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.773747 23779 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:36.774596 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:36.774812 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:36.775308 23928 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.775308 23858 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:36.775928 23858 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:36.776872 23858 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.778299 23858 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.782575 23928 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.783578 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:36.784328 25671 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:36.784752 25671 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:36.785526 25671 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.786592 23858 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.797487 23682 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.934894 25629 raft_consensus.cc:993] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.935472 25627 raft_consensus.cc:1081] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.937206 25614 raft_consensus.cc:993] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.937302 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:36.937965 23781 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:36.938310 25605 raft_consensus.cc:1081] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.938357 23781 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.942070 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:36.942632 23780 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:36.943037 23780 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.948559 23781 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.949815 23780 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:36.950150 23781 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:36.950973 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:36.951442 23858 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:36.951689 23858 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:36.952015 23780 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:36.952265 23858 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:36.953118 23858 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.953326 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:36.953836 23857 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.953922 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:36.954514 23928 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.959718 23858 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.960541 23857 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.961294 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:36.960855 23925 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:36.961730 23926 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:36.962050 23926 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:36.962630 23926 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:36.963523 23926 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:36.963742 23928 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.964995 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:36.965842 25671 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:36.966262 25671 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:36.966360 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:36.967116 25671 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:36.968416 23898 consensus_peers.cc:597] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8b83386d2fa4ed2a212fbe8873239ea. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:36.969715 25622 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:36.970153 23926 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:36.970903 25622 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:36.971807 25622 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:36.973178 23857 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:36.975893 25614 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:36.987280 23780 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:36.986811 23682 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.989329 25567 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:36.996649 25629 raft_consensus.cc:993] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.997282 25629 raft_consensus.cc:1081] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:36.999687 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:37.000319 23780 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:37.000721 23780 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.001354 25585 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.004602 23779 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.004897 23857 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.007496 23682 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:37.012167 23780 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.022269 23780 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.025731 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:37.026341 23926 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.028177 23682 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.029667 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:37.030254 23857 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:37.030586 23857 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:37.031332 23857 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.032605 23857 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.039849 23857 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:37.040334 23858 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:37.040938 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:37.041836 25622 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:37.042385 25622 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:37.042656 23926 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:37.043251 25622 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.072111 23682 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.078589 25629 raft_consensus.cc:993] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:37.080855 25551 raft_consensus.cc:1081] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:37.082688 25684 raft_consensus.cc:993] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:37.084918 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:37.085661 23780 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:37.086038 23780 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.090868 23779 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:37.092391 23780 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.092293 23926 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:37.092760 25684 raft_consensus.cc:1081] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:37.093765 25687 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.094862 23780 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.095768 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:37.096310 23926 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.096666 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:37.097515 23779 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:37.097877 23779 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:37.101243 23926 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
W20260709 12:13:37.097067 23824 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4369a8cdaaa45beac1359775fa54781. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:37.102438 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:37.104261 25687 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.104935 25622 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:37.105437 25622 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:37.106320 25622 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.110147 23779 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.110970 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:37.111644 23858 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:37.111940 23858 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:37.112985 23779 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.112803 23858 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.114398 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:37.114998 23856 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:37.115057 23858 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.116747 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:37.118325 23928 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:37.122238 23858 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:37.115343 23856 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:37.131228 25686 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.133572 23856 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.134861 23856 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:37.136392 23928 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:37.139559 23682 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.148075 23928 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.139559 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:37.144430 23777 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.143479 23856 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:37.161650 23926 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:37.162757 25622 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:37.164260 25622 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:37.165982 25622 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:37.165609 23681 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:37.182662 23898 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f5f207bbfed4f119db8464be0a5a8cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:37.190203 23856 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:37.190515 23777 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:37.192348 25585 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:37.197821 25585 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.206089 23682 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.213389 25567 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.215322 23779 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.219508 23856 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.234586 23928 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:37.241237 23681 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 4, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:37.244987 23898 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 956a6729b1f34c499670aa77b5896a53. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:37.245169 23856 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:37.245911 23779 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:37.246825 25585 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.248692 25585 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.264539 23926 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.273588 25623 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:37.275906 25605 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.279225 23858 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.285493 25671 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.288141 23856 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.320274 23681 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.346130 23780 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:37.422320 23926 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.424073 25623 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.424687 25703 ts_tablet_manager.cc:933] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.429843 23856 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:37.431494 25671 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.445760 25703 tablet_copy_client.cc:323] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:37.448547 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet c8b83386d2fa4ed2a212fbe8873239ea from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:37.449088 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet c8b83386d2fa4ed2a212fbe8873239ea from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-c8b83386d2fa4ed2a212fbe8873239ea
I20260709 12:13:37.449347 23857 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.450965 25660 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.479559 23926 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:37.484586 25660 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.493206 23858 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.494915 25660 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.528035 23926 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.530827 25660 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.495388 23943 tablet_copy_source_session.cc:215] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:37.537981 25703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b83386d2fa4ed2a212fbe8873239ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:37.559216 23857 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.561199 25660 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.563166 25703 tablet_copy_client.cc:806] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:37.563813 25703 tablet_copy_client.cc:670] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:37.568470 25703 tablet_copy_client.cc:538] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:37.582100 23926 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:37.583114 25703 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:37.586042 25671 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:37.605448 23857 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:37.618649 25660 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:37.624063 23926 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:37.625708 25671 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.639151 25716 ts_tablet_manager.cc:933] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:37.681324 25716 tablet_copy_client.cc:323] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:37.683455 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet c4369a8cdaaa45beac1359775fa54781 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:37.684157 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet c4369a8cdaaa45beac1359775fa54781 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-c4369a8cdaaa45beac1359775fa54781
I20260709 12:13:37.752632 25713 ts_tablet_manager.cc:933] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.758174 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:37.796284 25713 tablet_copy_client.cc:323] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:37.800228 23870 tablet_copy_source_session.cc:215] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:37.825778 25716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4369a8cdaaa45beac1359775fa54781. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:13:37.860150 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 93721.6us
I20260709 12:13:37.865473 25703 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:37.866916 25703 tablet_bootstrap.cc:492] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:37.868633 25723 ts_tablet_manager.cc:933] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:37.867734 25703 ts_tablet_manager.cc:1403] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.285s	user 0.054s	sys 0.041s
I20260709 12:13:37.874419 25703 raft_consensus.cc:359] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:37.887214 25703 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:37.891286 25723 tablet_copy_client.cc:323] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:13:37.887966 25703 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:37.898597 25703 ts_tablet_manager.cc:1434] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.003s
I20260709 12:13:37.909893 25716 tablet_copy_client.cc:806] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:37.912896 25716 tablet_copy_client.cc:670] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:37.935679 25716 tablet_copy_client.cc:538] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:37.946408 23943 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 7f5f207bbfed4f119db8464be0a5a8cf from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:37.947245 23943 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 7f5f207bbfed4f119db8464be0a5a8cf from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-7f5f207bbfed4f119db8464be0a5a8cf
I20260709 12:13:37.956939 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 956a6729b1f34c499670aa77b5896a53 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:13:37.957798 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 956a6729b1f34c499670aa77b5896a53 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-956a6729b1f34c499670aa77b5896a53
I20260709 12:13:37.971912 25716 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:37.972750 23942 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c8b83386d2fa4ed2a212fbe8873239ea received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:37.973423 23942 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c8b83386d2fa4ed2a212fbe8873239ea on tablet c8b83386d2fa4ed2a212fbe8873239ea with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:37.976444 23944 tablet_copy_source_session.cc:215] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:37.979817 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:37.980449 23856 raft_consensus.cc:606] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:37.971092 23943 tablet_copy_source_session.cc:215] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:37.985440 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:37.986054 23926 raft_consensus.cc:606] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:37.988926 25713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f5f207bbfed4f119db8464be0a5a8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:37.993415 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:37.994096 23926 raft_consensus.cc:606] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:37.997534 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:37.998104 23926 raft_consensus.cc:606] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:37.999795 25723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956a6729b1f34c499670aa77b5896a53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:38.001293 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:38.001902 23926 raft_consensus.cc:606] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:38.008018 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:38.010219 25175 raft_consensus.cc:1217] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:38.010651 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:38.011678 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:38.076248 25716 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:38.077543 25716 tablet_bootstrap.cc:492] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:38.078418 25716 ts_tablet_manager.cc:1403] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.107s	user 0.022s	sys 0.034s
I20260709 12:13:38.080699 25713 tablet_copy_client.cc:806] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:38.081457 25713 tablet_copy_client.cc:670] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:38.086585 25713 tablet_copy_client.cc:538] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:38.097424 25716 raft_consensus.cc:359] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.098742 25716 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:38.099743 25716 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.108139 25713 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:38.114694 25716 ts_tablet_manager.cc:1434] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.007s
I20260709 12:13:38.115459 25723 tablet_copy_client.cc:806] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:38.117928 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c4369a8cdaaa45beac1359775fa54781 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:38.118592 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c4369a8cdaaa45beac1359775fa54781 on tablet c4369a8cdaaa45beac1359775fa54781 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:38.124888 25723 tablet_copy_client.cc:670] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:38.137050 25723 tablet_copy_client.cc:538] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:38.143024 25721 raft_consensus.cc:993] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.144309 25725 raft_consensus.cc:1081] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.150825 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:38.151386 23780 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.154223 23780 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.163795 25723 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:38.177282 25713 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:38.178416 25713 tablet_bootstrap.cc:492] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:38.179350 25713 ts_tablet_manager.cc:1403] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.072s	user 0.031s	sys 0.008s
I20260709 12:13:38.183005 25713 raft_consensus.cc:359] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.184008 25713 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:38.184746 25713 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.190095 25713 ts_tablet_manager.cc:1434] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.005s
I20260709 12:13:38.189281 23780 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.202841 23780 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:38.208634 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:38.208806 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-7f5f207bbfed4f119db8464be0a5a8cf received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:38.209292 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-7f5f207bbfed4f119db8464be0a5a8cf on tablet 7f5f207bbfed4f119db8464be0a5a8cf with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:38.210121 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:38.210683 23926 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.209286 23857 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:38.213580 23857 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:38.214385 23857 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.215623 23857 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.220768 23926 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:38.222494 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:38.223359 25660 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:38.223573 25723 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:38.224620 25723 tablet_bootstrap.cc:492] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:38.225044 25660 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:38.225323 25723 ts_tablet_manager.cc:1403] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.062s	user 0.025s	sys 0.019s
I20260709 12:13:38.226776 23857 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:38.225950 25660 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.228775 25723 raft_consensus.cc:359] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.229656 25723 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:38.230443 25723 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:38.233513 25723 ts_tablet_manager.cc:1434] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:13:38.235796 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-956a6729b1f34c499670aa77b5896a53 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:13:38.236234 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-956a6729b1f34c499670aa77b5896a53 on tablet 956a6729b1f34c499670aa77b5896a53 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:38.237087 25175 raft_consensus.cc:1217] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:13:38.254208 23681 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.261315 25567 raft_consensus.cc:993] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.261876 25724 raft_consensus.cc:1081] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.263254 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:38.264034 23780 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.264423 23780 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.273237 23780 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.275098 23780 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:38.279182 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:38.279865 23926 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:38.280171 23926 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:38.281028 23926 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.282389 23926 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.276252 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:38.289335 23857 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.289526 23926 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:38.290782 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:38.291625 25623 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:38.292055 25623 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:38.292837 25623 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.296213 23857 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:38.305994 23681 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 4 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.364177 25567 raft_consensus.cc:993] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:38.364600 25724 raft_consensus.cc:1081] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:38.366024 23858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:13:38.366462 23858 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.366725 23858 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:38.375960 23858 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.377655 23858 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:38.378561 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:38.379168 23926 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:38.379514 23926 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:38.381825 23926 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.383124 23926 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:38.387831 23926 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:38.389756 23824 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:38.390547 25720 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:38.391032 25720 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:38.391887 25720 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.392647 25567 raft_consensus.cc:993] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.393107 25567 raft_consensus.cc:1081] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:38.394868 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:38.395368 23777 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.395737 23777 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.401895 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:38.406365 23780 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:38.402915 23777 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.410012 23777 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:38.411260 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:38.418519 23858 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.418269 23681 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.429703 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:38.430918 23926 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.431293 23926 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:38.432211 23926 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.433652 23926 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.435366 25585 raft_consensus.cc:993] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:38.435892 25585 raft_consensus.cc:1081] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:38.437630 25175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:13:38.438163 25175 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.438474 25175 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.441533 23926 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:38.445489 25175 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.448052 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:38.448647 23857 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.449183 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:38.449949 25623 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:38.450407 25623 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:38.451344 25623 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:38.451759 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:38.452359 23926 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:38.452636 23926 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:38.453336 23926 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.455220 25175 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:38.461943 23857 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:38.464980 25177 raft_consensus.cc:1217] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:38.454335 23926 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.473124 23780 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:38.474123 23858 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:38.482961 23926 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:38.487000 23680 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.489820 25119 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:38.491992 25669 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:38.490494 25614 raft_consensus.cc:1064] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:38.496804 25614 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.509788 25669 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:38.511116 25669 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.519271 23857 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:38.520167 25175 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:38.521697 25585 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:38.522526 25614 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:38.524219 23779 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:38.526599 25614 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:38.539244 23680 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:38.558040 25585 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.559872 23779 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.568907 25175 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.575369 23857 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.585633 23680 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:38.621023 25175 raft_consensus.cc:1217] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 12:13:38.688226 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 63727.4us
W20260709 12:13:38.689764 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 66165.7us
I20260709 12:13:38.698115 23926 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.708196 25177 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:38.712214 23858 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:38.719738 25614 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.722241 25605 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.734431 25724 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.744920 25174 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.745936 23857 raft_consensus.cc:2955] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.758929 23926 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:38.762770 23683 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 3 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:38.764320 23853 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:38.768882 23928 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:38.772939 25660 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.774971 25660 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.778647 25660 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.795006 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c8b83386d2fa4ed2a212fbe8873239ea with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:38.832911 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet c8b83386d2fa4ed2a212fbe8873239ea with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43412
I20260709 12:13:38.834156 25724 raft_consensus.cc:1064] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:38.838613 25751 tablet_replica.cc:333] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:13:38.841564 25724 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:38.843750 25751 raft_consensus.cc:2243] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:38.844992 25751 raft_consensus.cc:2272] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:38.848623 23853 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:38.856374 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 47313.1us
I20260709 12:13:38.862236 25751 ts_tablet_manager.cc:1916] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:38.869215 23780 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:38.876091 25752 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.898195 23928 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:38.898427 25629 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:13:38.919746 25751 ts_tablet_manager.cc:1929] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:13:38.920408 25751 log.cc:1199] T c8b83386d2fa4ed2a212fbe8873239ea P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/c8b83386d2fa4ed2a212fbe8873239ea
I20260709 12:13:38.929626 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet c8b83386d2fa4ed2a212fbe8873239ea (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:38.940577 23853 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:38.940820 25174 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:38.942777 25605 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.951313 23853 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:38.951845 23780 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:38.954473 25567 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.957751 25757 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.959915 25585 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.964871 25752 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.976559 25721 raft_consensus.cc:1064] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:38.979292 25721 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.025923 25175 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.028715 23925 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.028481 25605 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.038172 23781 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.039093 23926 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:39.043287 25759 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:39.050860 23853 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.058386 25759 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:39.072518 25174 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.080775 25623 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:39.081475 25758 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:39.088070 23680 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:39.090269 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:39.103394 23853 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:39.103889 23784 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.105197 25669 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:39.146732 23925 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:39.142851 25758 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.150707 25669 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:13:39.172825 23784 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.198925 25174 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.212808 23926 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.223405 23680 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.264289 25585 raft_consensus.cc:1064] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:39.266708 25585 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.269543 23925 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.283612 25175 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:39.288144 23781 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:39.294521 25614 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:39.290932 23853 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.298734 25605 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:39.303640 23784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:53932
I20260709 12:13:39.304286 23784 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:39.307657 23784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:53932
I20260709 12:13:39.308234 23784 raft_consensus.cc:606] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:39.311751 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:39.312381 23926 raft_consensus.cc:606] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:39.313793 25567 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.315809 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:39.316493 23926 raft_consensus.cc:606] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:39.318102 25660 raft_consensus.cc:993] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:39.323788 25660 raft_consensus.cc:1081] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:39.322870 25174 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:39.326524 25758 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:39.333542 23853 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:39.334161 23853 raft_consensus.cc:606] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:39.334734 25177 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.344944 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:39.346879 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:39.347772 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:39.345570 25758 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.353935 25174 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.357048 25724 raft_consensus.cc:993] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:39.357782 25724 raft_consensus.cc:1081] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:39.360615 25176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:13:39.361375 25176 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:39.361779 25176 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.376004 25175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:53494
I20260709 12:13:39.376765 25175 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:39.377140 25175 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.381141 23784 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:39.391358 23683 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.399013 25724 raft_consensus.cc:993] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:39.399665 25605 raft_consensus.cc:1081] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:39.403685 23857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:13:39.404260 23857 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:39.404597 23857 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:39.420845 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet c4369a8cdaaa45beac1359775fa54781 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45604
I20260709 12:13:39.429049 25759 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:39.434003 23784 raft_consensus.cc:2955] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.437530 23680 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 6, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.439298 23857 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.451376 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:39.451469 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f5f207bbfed4f119db8464be0a5a8cf with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:39.452093 23926 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:39.452425 23926 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:39.453220 23926 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.454288 23926 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:39.460433 25176 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.463797 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:39.464372 23853 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.467047 23857 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:39.474172 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f5f207bbfed4f119db8464be0a5a8cf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:39.473166 23853 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:39.475888 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:39.476902 25773 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:39.477439 25773 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:39.478327 25773 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.489692 23680 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:39.491500 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c4369a8cdaaa45beac1359775fa54781 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:39.492323 25780 tablet_replica.cc:333] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:13:39.493511 25780 raft_consensus.cc:2243] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:39.494308 25780 raft_consensus.cc:2272] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:39.464294 23926 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:39.497140 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:39.498013 25759 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:39.498170 25780 ts_tablet_manager.cc:1916] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:39.498515 25759 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:39.499786 25759 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.508862 23776 raft_consensus.cc:2955] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.533576 25176 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:39.534451 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:39.535059 23926 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:39.535346 23926 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:39.535986 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:39.539667 23784 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:39.540992 25781 tablet_replica.cc:333] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:39.542737 25781 raft_consensus.cc:2243] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:39.531142 25175 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.564545 23926 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.567519 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:39.568264 23857 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.573232 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:39.573495 25175 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:39.573905 23778 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:39.574236 23778 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:39.575203 23778 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.582561 23778 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:39.537314 23681 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.590477 23926 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.605350 25781 raft_consensus.cc:2272] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:39.606386 25176 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.609128 23858 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.611356 25781 ts_tablet_manager.cc:1916] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:39.610875 25724 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:39.621264 25605 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:39.653935 25780 ts_tablet_manager.cc:1929] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:39.654564 25780 log.cc:1199] T c4369a8cdaaa45beac1359775fa54781 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/c4369a8cdaaa45beac1359775fa54781
W20260709 12:13:39.665378 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 50614.6us
I20260709 12:13:39.668581 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet c4369a8cdaaa45beac1359775fa54781 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:39.678504 23776 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:39.616236 23784 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:39.687075 25605 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:39.687969 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:13:39.690776 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:39.691205 23926 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:39.691995 25773 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:39.693228 25752 raft_consensus.cc:993] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:39.704202 25752 raft_consensus.cc:1081] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:39.704118 25773 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:39.706175 25773 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.687481 23778 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:13:39.713245 23853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:13:39.713840 23853 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:39.713896 25781 ts_tablet_manager.cc:1929] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:13:39.714236 23853 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.714587 25781 log.cc:1199] T 7f5f207bbfed4f119db8464be0a5a8cf P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/7f5f207bbfed4f119db8464be0a5a8cf
I20260709 12:13:39.716879 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 7f5f207bbfed4f119db8464be0a5a8cf (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:39.722505 23853 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.726222 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:39.726869 23778 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:39.727172 23778 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:39.745220 23853 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:39.738889 23778 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.747141 23778 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.748360 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:39.754101 23926 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.759912 23778 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:39.778136 25614 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.791349 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:39.790683 23778 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.793429 25758 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:39.793965 25758 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:39.794792 25758 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.787338 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 7, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:39.787374 23926 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:39.821642 23857 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.842801 23682 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.861667 25176 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.870942 23681 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.875331 25758 raft_consensus.cc:993] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:39.875865 25758 raft_consensus.cc:1081] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:39.877642 23778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:39.878177 23778 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:39.878500 23778 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.885399 23778 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.887259 23778 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:39.890220 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:39.890805 23858 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:39.891122 23858 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:39.892223 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:39.892771 23926 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.893452 23858 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.894585 23858 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.901842 23928 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:39.902697 23926 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:39.904059 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:39.904918 25749 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:39.905400 25749 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:39.906230 25749 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:39.910362 25176 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:39.910521 23858 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:39.913904 23776 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:39.914381 25724 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:39.924154 25724 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:39.927641 23681 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.933032 25614 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.933795 25176 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.960531 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 956a6729b1f34c499670aa77b5896a53 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:39.962597 23776 raft_consensus.cc:2955] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.964282 23681 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 5, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 2 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:39.975399 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 956a6729b1f34c499670aa77b5896a53 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56638
I20260709 12:13:39.976444 25781 tablet_replica.cc:333] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:13:39.977272 25781 raft_consensus.cc:2243] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:39.977897 25781 raft_consensus.cc:2272] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:39.981753 25781 ts_tablet_manager.cc:1916] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:39.997799 25781 ts_tablet_manager.cc:1929] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:39.998147 25781 log.cc:1199] T 956a6729b1f34c499670aa77b5896a53 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/956a6729b1f34c499670aa77b5896a53
I20260709 12:13:39.999758 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 956a6729b1f34c499670aa77b5896a53 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:13:40.000666 23858 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:40.002717 25773 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:40.012812 23926 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:40.014257 25773 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:40.033442 23776 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:40.034801 25759 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:40.062083 23926 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:40.063630 25758 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:40.277300 23776 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:13:40.278589 25796 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:40.292821 23857 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:13:40.294281 25773 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:40.359352 23928 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:40.360666 25758 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:40.369987 23926 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:40.380152 23784 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:40.383996 25789 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:40.393426 25629 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:40.406970 23857 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:40.408522 25749 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:40.777565 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:40.813570 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "102493ff11494b009f79963bb57be511"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:40.814100 23858 raft_consensus.cc:606] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:40.816401 23784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:53932
I20260709 12:13:40.816841 23784 raft_consensus.cc:606] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:40.819011 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:40.819455 23926 raft_consensus.cc:606] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:40.822045 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:40.822569 23858 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:40.824860 23926 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:40.825330 23926 raft_consensus.cc:606] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:40.826294 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:40.827414 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:40.828022 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:40.998785 25758 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:40.999501 25759 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.001354 25176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44500
I20260709 12:13:41.001917 25176 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.002254 25176 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.010109 25176 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.012961 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:41.013554 25176 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:41.015023 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:41.015627 23858 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:41.015929 23858 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:41.013574 23776 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.017001 23858 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.018249 23858 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.024245 23776 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.025826 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:41.026814 25773 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:41.027287 25773 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:41.029667 23858 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.032461 25773 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.047197 23683 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 1 to 2, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 2 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:41.060060 23858 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:41.070959 23926 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:41.071003 23824 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e60a1c6520545729d27474d8f237ce5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.072619 25629 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:41.074033 25758 raft_consensus.cc:993] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.074533 25758 raft_consensus.cc:1081] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.078889 23776 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:41.080160 25176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "102493ff11494b009f79963bb57be511"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44500
I20260709 12:13:41.080302 25759 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:41.080674 25176 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.081044 25176 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:41.084189 25629 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.086290 23928 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.088805 25176 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.091653 25176 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:41.094581 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:41.095207 23776 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:41.104084 23683 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:41.109032 25749 raft_consensus.cc:993] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.109647 25789 raft_consensus.cc:1081] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.111665 25175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:53494
I20260709 12:13:41.112172 25175 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.112428 25175 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.118572 25175 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.119433 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:41.120105 23858 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:41.120434 23858 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:41.121176 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:41.122159 23778 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:13:41.122408 23731 auto_rebalancer.cc:629] Failed to schedule move for tablet 34e5e0c10245477d9010ece72d08ae51: Remote error: Service unavailable: leader transfer in progress
I20260709 12:13:41.122457 23778 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:41.123303 25175 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:41.123885 23778 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.133692 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:41.134315 23928 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.138980 23776 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:41.140959 25114 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:41.144309 25773 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:41.146198 25773 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:41.148121 23778 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.152029 23857 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:41.147328 25773 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.122406 23858 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.155496 23928 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.166848 23784 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.176601 23928 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:41.179138 23776 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:41.178970 25629 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:13:41.184566 23858 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:41.184474 25758 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:41.190614 23778 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.200274 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:41.201318 25669 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:41.201845 25669 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:41.201937 23858 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
W20260709 12:13:41.203267 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67876.5us
W20260709 12:13:41.203984 23824 consensus_peers.cc:597] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de3c70defd864769b26b52b7f36548cd. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.203017 25669 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.217794 25585 raft_consensus.cc:993] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.218341 25614 raft_consensus.cc:1081] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:41.222074 25176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:13:41.222646 25176 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.223021 25176 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.225526 23681 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:41.225564 25758 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.229486 25176 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.232169 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:41.232831 23777 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.232098 23776 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.239392 23777 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.240785 25114 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:41.241760 25773 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:41.243634 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:41.243079 23681 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 2 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:41.244189 23926 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:41.244551 23926 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:41.245288 23926 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.246315 23926 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.257171 25773 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:41.256995 23778 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:41.258204 25773 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:41.259049 25819 raft_consensus.cc:993] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:41.259522 25819 raft_consensus.cc:1081] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:41.261785 25176 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:41.262609 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:41.263198 23777 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.263617 23777 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:41.263001 23683 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 2 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:41.266772 23926 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:41.273830 23777 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:41.276041 23777 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:41.277303 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:41.277992 23925 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:41.278278 23925 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:41.279100 23925 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:41.280426 23925 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:41.299034 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:41.301136 23857 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:41.310981 23925 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:41.312392 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:41.320531 25789 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:41.321115 25789 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:41.322060 25789 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:41.335987 25824 log.cc:927] Time spent T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:13:41.358182 23857 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:41.871208 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:41.947365 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.009s
W20260709 12:13:41.950455 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.002s	sys 0.009s
W20260709 12:13:41.950764 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 590765us
I20260709 12:13:41.949433 23776 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:41.957100 25837 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:41.959751 23776 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:13:41.990479 23778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.008s	sys 0.013s
W20260709 12:13:41.990895 23778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.008s	sys 0.013s
W20260709 12:13:42.011318 25789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.009s	sys 0.005s
W20260709 12:13:42.011775 25789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.010s	sys 0.005s
I20260709 12:13:42.193262 23933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:42.194108 23933 raft_consensus.cc:606] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:42.269548 25834 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.273466 25834 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.277688 23933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:42.278460 23933 raft_consensus.cc:606] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:42.281733 25834 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:42.347307 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.026s	user 0.021s	sys 0.019s
W20260709 12:13:42.347820 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.023s	sys 0.019s
W20260709 12:13:42.348088 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 1.02685e+06us
I20260709 12:13:42.351068 25827 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.351408 25837 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:13:42.351680 25827 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.362509 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:42.363150 23934 raft_consensus.cc:606] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:42.363547 25827 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:42.399861 23934 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:42.405182 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:42.405483 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:42.408849 25827 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.409528 25827 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.412197 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
W20260709 12:13:42.412153 23749 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6d3d4f634284532bbf6ad7d4ac19214. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:42.414165 25827 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.413000 23934 raft_consensus.cc:606] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:42.419862 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08000de5c0 after lost signal to thread 22632
W20260709 12:13:42.423418 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08000de180 after lost signal to thread 23717
W20260709 12:13:42.423870 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3e20 after lost signal to thread 23731
W20260709 12:13:42.425064 23806 debug-util.cc:398] Leaking SignalData structure 0x7b080020fa20 after lost signal to thread 23809
W20260709 12:13:42.425350 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec480 after lost signal to thread 23810
I20260709 12:13:42.425571 23776 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:42.433071 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
W20260709 12:13:42.434120 23825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.012s	sys 0.017s
I20260709 12:13:42.434501 23934 raft_consensus.cc:606] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:42.438795 25844 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.439388 25833 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.440068 25833 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:42.438946 23825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.012s	sys 0.017s
I20260709 12:13:42.441534 25773 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:13:42.466415 23781 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:42.471156 23934 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:42.476240 25853 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.476830 25853 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.478912 25853 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.479954 23858 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:42.481316 23853 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:42.481541 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:42.483510 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:42.484381 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:42.494424 25833 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.497723 25838 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:42.501946 25839 raft_consensus.cc:993] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:13:42.502684 23806 debug-util.cc:398] Leaking SignalData structure 0x7b0800348260 after lost signal to thread 23884
I20260709 12:13:42.506633 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.444518 25837 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.520752 25833 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.521464 25833 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.523818 25839 raft_consensus.cc:1081] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:42.526270 25749 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.526895 25749 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.529192 25749 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.532570 25796 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:42.534874 25796 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.535409 25796 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.538393 25796 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.541326 25796 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:42.541441 23681 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 1 to 2, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 2 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:42.545536 23778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:42.546140 23778 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:42.546496 23778 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.515946 25819 raft_consensus.cc:993] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:42.439489 25844 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.554340 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:13:42.566439 23778 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.548310 25837 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.577885 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.578219 25837 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.578835 25833 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:42.579581 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.583101 23778 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.584553 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.594463 25585 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.594997 25585 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:42.595654 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 95909.9us
I20260709 12:13:42.597611 25614 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.598100 25614 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.602041 23778 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:42.561774 25839 raft_consensus.cc:1081] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:42.610020 25585 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.620169 25614 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.622732 25614 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.623306 25614 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:42.631867 25614 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.595942 25819 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.636058 25614 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.651095 25614 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.636570 25835 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:42.636857 23806 debug-util.cc:398] Leaking SignalData structure 0x7b0800351000 after lost signal to thread 23958
I20260709 12:13:42.637101 25629 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:42.598163 25844 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.654835 25629 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.635004 25854 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:13:42.637763 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.638213 25843 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.638610 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:42.638070 25861 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:42.639417 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.638115 25851 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:42.643777 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.646026 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.685038 23777 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:42.621047 25838 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.662343 23853 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:42.663050 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 96161us
I20260709 12:13:42.599555 25773 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 12:13:42.669512 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 72455.2us
I20260709 12:13:42.694439 25863 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:42.697593 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:42.699455 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.699831 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.725674 25614 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.699842 25860 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.700500 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.717093 25836 raft_consensus.cc:993] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:42.722994 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.700471 23776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:13:42.730391 25629 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.638633 25845 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:42.760288 23776 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:42.760488 23781 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:42.773499 23776 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.771306 23853 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:42.798138 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:42.806831 25856 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:42.807482 25856 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.809645 25856 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:42.722959 25843 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.810196 25818 raft_consensus.cc:1081] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:42.731168 25838 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.656581 25819 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.812004 25818 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.813954 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:42.819764 23776 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.823750 25819 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.823856 25838 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.760262 25860 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.825704 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:42.826855 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65943.5us
I20260709 12:13:42.826426 23927 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:42.831851 23776 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:42.834612 25614 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:42.835080 25614 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:42.835882 25614 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.843699 25614 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.846037 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:42.831822 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 144591us
I20260709 12:13:42.841715 25855 raft_consensus.cc:993] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:42.842617 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:42.847246 25853 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:42.841109 25864 ts_tablet_manager.cc:933] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:42.850769 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.853242 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.725960 25851 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.859541 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:42.884382 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:42.885195 23934 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:42.885561 23934 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:42.880399 23806 debug-util.cc:398] Leaking SignalData structure 0x7b0800333700 after lost signal to thread 25207
I20260709 12:13:42.854312 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:42.661949 25835 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:42.873225 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 185596us
I20260709 12:13:42.935817 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:42.878719 25855 raft_consensus.cc:1081] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:13:42.761929 25845 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.930749 25833 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.878702 23933 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:42.721763 25861 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.939883 23926 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:42.930415 25789 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:42.953543 23934 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:42.956727 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.957036 23934 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.959899 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.941291 25173 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:13:42.879133 25853 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.990072 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:42.821251 25818 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.962225 25833 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.998694 25173 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:42.961372 25789 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:42.999183 25173 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.000933 25789 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.004900 25833 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.005409 25827 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.006243 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.020859 25843 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:43.028661 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.029357 25845 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.049013 25861 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:42.872152 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 260934us
I20260709 12:13:43.054073 25851 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.054950 25818 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:43.059018 25835 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:43.064999 25174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:13:43.065632 25174 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:43.065977 25174 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.076408 25174 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.083841 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:43.090039 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.084530 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:43.092468 25174 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.100862 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:43.102121 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.104627 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.116460 23934 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:43.125391 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.125989 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:43.101560 25172 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.153102 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:43.153828 23853 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.150990 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.169548 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.151365 25844 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:43.204182 25858 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:43.211910 25860 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:43.266103 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.283986 25174 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:43.286144 25843 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:43.288069 25860 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:43.288718 25860 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:43.290230 25860 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.292071 25860 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.288522 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:43.299149 25854 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:43.299758 25854 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:43.300268 25854 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:43.329648 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.337847 25853 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:43.329452 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 203536us
I20260709 12:13:43.341409 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:43.342242 25853 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:43.373636 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.382323 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:43.389339 23853 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:43.393630 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.395974 25614 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:43.396987 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:43.399581 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.401297 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:43.410764 25845 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.411520 25845 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.414551 25845 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:43.417536 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:43.422147 25867 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.424952 25119 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:43.437414 25796 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:13:43.377708 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 89340.4us
I20260709 12:13:43.457216 25834 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:43.458173 25834 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:43.465620 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.000s	sys 0.040s
W20260709 12:13:43.466387 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.001s	sys 0.040s
I20260709 12:13:43.479084 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.463241 25860 raft_consensus.cc:993] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:43.498652 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:43.500999 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:43.454476 25796 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:43.507269 25585 raft_consensus.cc:1081] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:13:43.503304 25629 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:43.504422 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.505159 25614 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:43.511457 25614 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:43.536098 25832 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:43.463138 25864 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.016s	sys 0.029s
I20260709 12:13:43.553555 25614 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:43.572530 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57251.3us
I20260709 12:13:43.575558 25834 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:43.576671 25796 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:43.565237 25173 consensus_meta.cc:287] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.566s	user 0.001s	sys 0.012s
W20260709 12:13:43.572230 25864 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.016s	sys 0.029s
I20260709 12:13:43.592387 23857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:13:43.621814 23857 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:43.622326 23857 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.627425 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.642043 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:43.650177 25876 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:13:43.666617 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:43.667243 23926 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:43.589108 25173 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.670818 25174 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:43.686794 25172 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.001s
I20260709 12:13:43.687544 25172 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:43.709514 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:43.725889 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:43.674542 25173 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.730945 23857 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:43.674479 25854 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.734336 25614 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.733269 25818 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:13:43.736770 23857 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:43.737915 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:43.741408 25869 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:43.744993 25614 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:43.745494 25614 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:43.746232 25614 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.747363 25614 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.571821 25832 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.725766 25864 tablet_copy_client.cc:323] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
W20260709 12:13:43.752097 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.754251 23933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.021s	sys 0.049s
W20260709 12:13:43.754606 23933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.021s	sys 0.049s
W20260709 12:13:43.755095 25176 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.028s	sys 0.041s
W20260709 12:13:43.758176 25176 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.029s	sys 0.041s
I20260709 12:13:43.744943 25874 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:43.764243 23750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.013s	sys 0.029s
W20260709 12:13:43.766410 23750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.013s	sys 0.029s
I20260709 12:13:43.753180 25818 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:43.767623 23932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.027s	sys 0.023s
I20260709 12:13:43.769563 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:43.769248 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.020s	sys 0.051s
W20260709 12:13:43.791128 23932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.027s	sys 0.030s
W20260709 12:13:43.788473 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.020s	sys 0.056s
W20260709 12:13:43.792053 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 862022us
I20260709 12:13:43.792840 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:43.768689 25834 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.795821 25834 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:43.791867 25773 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.036s	sys 0.064s
W20260709 12:13:43.808331 25773 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.037s	sys 0.064s
I20260709 12:13:43.813097 25834 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:43.813277 25872 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:43.814039 25872 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.803825 25789 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:43.793771 25818 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:43.777621 25838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.021s	sys 0.053s
I20260709 12:13:43.856702 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:43.861039 23855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.016s	sys 0.058s
I20260709 12:13:43.849802 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:43.906837 23855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.016s	sys 0.059s
I20260709 12:13:43.911319 23933 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:43.846328 25818 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.916705 25614 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:43.847679 25838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.022s	sys 0.055s
I20260709 12:13:43.919332 25614 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:43.920082 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70303.2us
W20260709 12:13:43.923321 25880 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:13:43.921335 25870 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:43.947283 25614 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:43.976873 23857 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:43.978981 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 474782us
W20260709 12:13:44.037853 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46182.5us
I20260709 12:13:43.769354 25874 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:43.967185 25854 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:43.984625 25861 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:43.971578 25887 ts_tablet_manager.cc:933] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:43.974045 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 81) took 1276 ms. Trace:
I20260709 12:13:44.042972 25585 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:43.949324 25884 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:43.944895 25855 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:43.970815 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:44.074851 25614 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:43.974828 25832 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:43.984465 25851 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.081702 25614 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:43.966737 25872 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:44.081676 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:44.077361 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:44.088474 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 394494us
I20260709 12:13:44.036897 25789 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.097837 25614 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:44.125608 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 4e60a1c6520545729d27474d8f237ce5 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:43.967602 25870 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.101977 25872 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.135604 25845 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.130124 25789 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.136863 25845 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.138732 25891 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:43.970966 25834 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:44.088197 25669 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.038s
I20260709 12:13:44.139523 25891 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:44.140300 25669 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.001s	sys 0.039s
I20260709 12:13:44.140060 25845 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:44.141389 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.128185 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:44.077196 25835 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.127612 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:43.973402 25858 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:43.972757 25846 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.135763 25872 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.140076 25849 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.096429 25832 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:44.073609 25878 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.013s	sys 0.063s
I20260709 12:13:44.073107 23932 rpcz_store.cc:276] 0709 12:13:42.697417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.698265 (+   848us) service_pool.cc:224] Handling call
0709 12:13:42.698561 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.834931 (+136370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.835497 (+   566us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.836004 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:13:43.791558 (+955554us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:43.791629 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.791696 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.973806 (+182110us) raft_consensus.cc:1685] finished
0709 12:13:43.973828 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.973975 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27273,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":29365,"thread_start_us":1090080,"threads_started":2,"wal-append.queue_time_us":956457}
I20260709 12:13:44.146095 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:44.128237 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:44.080967 25861 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.129222 25844 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.128302 25885 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.144124 25845 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.111860 25851 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.153316 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 4e60a1c6520545729d27474d8f237ce5 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-4e60a1c6520545729d27474d8f237ce5
I20260709 12:13:44.184803 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:44.155424 25789 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:44.170470 25878 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.016s	sys 0.066s
I20260709 12:13:44.172526 25858 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.175194 25844 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.167088 25846 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.084569 25855 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.155190 25834 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:44.154577 25835 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.210856 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:44.213281 25889 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:13:44.166075 25849 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:13:44.239775 25837 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.020s	sys 0.057s
I20260709 12:13:44.171911 25832 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.186179 25845 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:44.326668 25113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.028s	sys 0.025s
W20260709 12:13:44.327075 25113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.029s	sys 0.025s
W20260709 12:13:44.245721 25837 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.020s	sys 0.057s
W20260709 12:13:44.331859 25796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.016s	sys 0.012s
W20260709 12:13:44.331948 23853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.010s	sys 0.018s
W20260709 12:13:44.437276 23853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.011s	sys 0.018s
W20260709 12:13:44.327338 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 564490us
W20260709 12:13:44.333463 23749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.396s	user 0.043s	sys 0.043s
W20260709 12:13:44.439658 23749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.502s	user 0.043s	sys 0.043s
W20260709 12:13:44.440016 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.5442e+06us
I20260709 12:13:44.440966 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:44.442176 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:44.324761 25773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.008s	sys 0.043s
I20260709 12:13:44.364327 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 85) took 1277 ms. Trace:
I20260709 12:13:44.180186 25885 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.375780 25894 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:44.391101 25874 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.394232 25845 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.394393 25861 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:44.397745 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.401290 25849 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:44.402756 25846 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.404981 25844 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:44.327284 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116579us
I20260709 12:13:44.407431 25872 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:44.466048 25773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.008s	sys 0.044s
I20260709 12:13:44.415189 23870 tablet_copy_source_session.cc:215] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:44.416796 25855 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:44.371213 25858 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.335903 25870 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:44.426332 25819 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:44.428527 25818 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.362212 25860 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.464242 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:44.462965 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.586s (SENT) [suppressed 19 similar messages]
I20260709 12:13:44.484524 25789 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:44.489912 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.489089 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.409332 25835 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:44.466136 23855 rpcz_store.cc:276] 0709 12:13:43.087073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.087197 (+   124us) service_pool.cc:224] Handling call
0709 12:13:43.100025 (+ 12828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:43.100382 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:43.100411 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:43.103901 (+  3490us) log.cc:844] Serialized 26 byte log entry
0709 12:13:43.216659 (+112758us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b1800177e80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:13:43.907400 (+690741us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:43.907446 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.907546 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.356188 (+448642us) raft_consensus.cc:1685] finished
0709 12:13:44.356211 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.364258 (+  8047us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":14646,"spinlock_wait_cycles":5878016,"thread_start_us":690480,"threads_started":1,"wal-append.queue_time_us":966188}
I20260709 12:13:44.495955 25890 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:44.502558 25834 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.503983 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:44.500484 25796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.016s	sys 0.012s
W20260709 12:13:44.638703 25884 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.031s	sys 0.043s
I20260709 12:13:44.652868 23932 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:13:44.653314 25884 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.035s	sys 0.047s
I20260709 12:13:44.653537 23928 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
W20260709 12:13:44.654838 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 191872us
I20260709 12:13:44.652848 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.656018 23925 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:13:44.596052 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.014s	sys 0.026s
W20260709 12:13:44.657152 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.014s	sys 0.029s
I20260709 12:13:44.653136 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.658512 23933 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:44.652778 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.660192 23934 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:44.513186 25851 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.497913 25854 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:44.689885 23925 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:44.692209 25899 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.644s	user 0.006s	sys 0.008s
I20260709 12:13:44.692124 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.650257 25838 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.653998 25884 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:44.695819 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.703207 23933 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:44.530320 25891 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.660723 25867 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:44.668859 25870 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:44.705546 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.712369 25867 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.704296 25854 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.545137 25819 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.707072 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:44.750974 23934 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:44.710646 25883 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.711577 25895 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.675935 25860 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.753036 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:44.754113 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 48753.5us
I20260709 12:13:44.682811 25872 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.707911 25838 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.756827 23824 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:44.766901 23750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.009s	sys 0.042s
W20260709 12:13:44.767226 23750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.009s	sys 0.042s
I20260709 12:13:44.713744 25173 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:44.708782 25832 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:44.769044 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:44.793344 23932 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:44.753752 25870 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:44.767457 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.799616 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.759379 25891 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:44.770931 25789 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:44.807996 25789 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.810309 25789 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.805783 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.821014 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.823829 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:44.827755 25871 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:13:44.785100 25846 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:44.837975 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:44.838663 23784 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:44.764976 25872 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.820250 25851 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:44.850358 23926 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.018s	sys 0.023s
I20260709 12:13:44.812760 25832 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:44.815728 25891 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.807432 25870 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:44.820510 25844 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:44.822556 25858 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:44.765461 25883 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:44.870281 25885 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.870443 23930 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:44.872182 25901 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:44.881915 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:44.882844 23778 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
W20260709 12:13:44.861296 23926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.018s	sys 0.023s
I20260709 12:13:44.834277 25861 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:44.885496 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 87) took 1032 ms. Trace:
I20260709 12:13:44.841018 25846 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.924490 23776 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:44.873745 25832 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:44.890136 25867 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:44.929090 25838 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:44.769948 25895 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:44.939720 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:44.949308 23778 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
W20260709 12:13:44.949606 23858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.028s	sys 0.018s
W20260709 12:13:44.949954 23858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.028s	sys 0.018s
I20260709 12:13:44.940554 25861 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.952106 25861 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:44.847422 25585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.050s	sys 0.073s
I20260709 12:13:44.861143 25851 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.957710 25851 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:44.857571 25865 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:44.866683 25891 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.866037 25844 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.868664 25858 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:44.965075 25585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.051s	sys 0.073s
W20260709 12:13:44.977433 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.015s	sys 0.013s
W20260709 12:13:44.977703 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.015s	sys 0.013s
W20260709 12:13:44.983042 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.021s	sys 0.048s
W20260709 12:13:44.983508 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.021s	sys 0.048s
W20260709 12:13:44.984548 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.984992 25585 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:44.986016 25585 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:44.989095 23854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.013s	sys 0.016s
W20260709 12:13:44.989471 23854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.013s	sys 0.016s
I20260709 12:13:44.943455 25846 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:44.872944 25901 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:44.949440 25878 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.009s	sys 0.036s
W20260709 12:13:45.055353 25878 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.009s	sys 0.036s
I20260709 12:13:44.942332 25818 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:45.072645 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 609662us
W20260709 12:13:44.986191 25669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.024s	sys 0.030s
W20260709 12:13:45.074206 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.008s	sys 0.047s
W20260709 12:13:45.074676 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.008s	sys 0.047s
W20260709 12:13:45.075006 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 839987us
I20260709 12:13:45.074295 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:45.076727 23856 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:13:45.058946 23779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.008s	sys 0.008s
W20260709 12:13:45.078395 23779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.008s	sys 0.008s
I20260709 12:13:44.976873 25858 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:44.963737 25879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.020s	sys 0.030s
I20260709 12:13:45.003342 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:44.973935 25844 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:44.991171 25865 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:45.085100 25669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.024s	sys 0.030s
W20260709 12:13:45.121351 23853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.000s	sys 0.019s
W20260709 12:13:44.999413 25887 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.021s	sys 0.020s
I20260709 12:13:44.940887 23926 rpcz_store.cc:276] 0709 12:13:43.853280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.859764 (+  6484us) service_pool.cc:224] Handling call
0709 12:13:43.860093 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.860288 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:44.048615 (+188327us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.885130 (+836515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.885189 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.885238 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.885306 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.885432 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188281,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":5182,"thread_start_us":1024046,"threads_started":2,"wal-append.queue_time_us":802025}
W20260709 12:13:45.140223 23853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.000s	sys 0.020s
I20260709 12:13:44.999739 25901 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:45.074102 25855 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.864365 25884 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.088124 25883 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:45.161576 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:45.139812 25879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.023s	sys 0.032s
I20260709 12:13:45.164428 25835 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:45.166230 25905 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
W20260709 12:13:45.141676 25887 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.021s	sys 0.020s
I20260709 12:13:45.167652 25853 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:45.129122 25837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.006s	sys 0.004s
I20260709 12:13:45.196681 25835 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.201025 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.201653 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:45.251617 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50032.4us
W20260709 12:13:45.250830 25911 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:13:45.203462 25856 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:45.253186 25796 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:45.253681 25796 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.254076 25796 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.203522 25836 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.255321 25836 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.203704 25853 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.165550 25855 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:45.267127 25904 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.526s	user 0.002s	sys 0.001s
I20260709 12:13:45.269318 25849 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:45.269846 25849 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:45.270731 25849 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.272579 25849 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.274626 23931 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:45.275127 23931 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:45.163229 25871 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:45.276059 23931 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.277069 25871 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.277586 23931 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:45.246726 25921 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
I20260709 12:13:45.171221 25865 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.252542 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:45.291821 25669 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:45.292356 25669 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:45.199469 25887 tablet_copy_client.cc:323] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:13:45.253288 25856 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.292738 25669 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.247150 25860 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:45.304533 25837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.007s	sys 0.004s
W20260709 12:13:45.390355 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 275638us
I20260709 12:13:45.398365 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 47) took 1285 ms. Trace:
I20260709 12:13:45.398964 23779 rpcz_store.cc:276] 0709 12:13:43.853385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.853530 (+   145us) service_pool.cc:224] Handling call
0709 12:13:43.853869 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.854080 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:44.240157 (+386077us) log.cc:844] Serialized 14 byte log entry
0709 12:13:45.138783 (+898626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:45.138844 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:45.138896 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.138963 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.139088 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59905,"prepare.queue_time_us":537636,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":88,"raft.queue_time_us":73411,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":34805,"thread_start_us":1222729,"threads_started":2,"wal-append.queue_time_us":925397}
I20260709 12:13:45.402343 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 788) took 1347 ms. Trace:
W20260709 12:13:45.402722 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.001s	sys 0.024s
I20260709 12:13:45.402894 23858 rpcz_store.cc:276] 0709 12:13:44.055013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.056539 (+  1526us) service_pool.cc:224] Handling call
0709 12:13:44.056852 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.057081 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.057115 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.082573 (+ 25458us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b2c00000448. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:13:44.089021 (+  6448us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.950309 (+861288us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:44.950351 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.950430 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.401565 (+451135us) raft_consensus.cc:1685] finished
0709 12:13:45.401604 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.402077 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24771968,"thread_start_us":805848,"threads_started":1,"wal-append.queue_time_us":1042718}
I20260709 12:13:45.170423 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 787) took 1115 ms. Trace:
W20260709 12:13:45.413488 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.012s	sys 0.036s
W20260709 12:13:45.414067 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.341s	user 0.012s	sys 0.037s
I20260709 12:13:45.425072 25885 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:45.426273 25885 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:45.403215 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.001s	sys 0.024s
W20260709 12:13:45.432443 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 602732us
I20260709 12:13:45.411581 23854 rpcz_store.cc:276] 0709 12:13:44.054699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.055114 (+   415us) service_pool.cc:224] Handling call
0709 12:13:44.055635 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.055995 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.056023 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.083201 (+ 27178us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b2c00000448. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:13:44.083915 (+   714us) log.cc:844] Serialized 499 byte log entry
0709 12:13:44.989795 (+905880us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.989835 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.989900 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.170112 (+180212us) raft_consensus.cc:1685] finished
0709 12:13:45.170132 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.170357 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26300672,"thread_start_us":905569,"threads_started":1,"wal-append.queue_time_us":1084586}
I20260709 12:13:45.470661 25907 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.497049 25912 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:45.498111 25912 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.502000 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 162) took 3454 ms. Trace:
W20260709 12:13:45.482709 23749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.003s	sys 0.032s
I20260709 12:13:45.471757 25907 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.508937 25929 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:45.510145 25885 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.411801 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:45.510844 23749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.004s	sys 0.032s
I20260709 12:13:45.521265 25906 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.510020 25929 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:45.524394 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 846705us
W20260709 12:13:45.529143 25853 net_util.cc:396] Time spent resolve address for 127.22.25.65: real 0.253s	user 0.010s	sys 0.004s
I20260709 12:13:45.529172 25872 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.532207 25819 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.534643 25895 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.522344 25906 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:45.525363 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 388217us
I20260709 12:13:45.539613 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 270) took 3096 ms. Trace:
W20260709 12:13:45.539431 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 449078us
I20260709 12:13:45.502589 23681 rpcz_store.cc:276] 0709 12:13:42.047480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.047633 (+   153us) service_pool.cc:224] Handling call
0709 12:13:45.501714 (+3454081us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:42.596730 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:42.597312 (+   582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:42.597345 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:13:42.597363 (+    18us) tablet.cc:662] Decoding operations
0709 12:13:42.597695 (+   332us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:42.597774 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:42.597801 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:42.650066 (+ 52265us) tablet.cc:701] Row locks acquired
0709 12:13:42.653284 (+  3218us) write_op.cc:260] PREPARE: finished
0709 12:13:42.684226 (+ 30942us) write_op.cc:270] Start()
0709 12:13:42.730994 (+ 46768us) write_op.cc:276] Timestamp: P: 1783599222663995 usec, L: 0
0709 12:13:42.758726 (+ 27732us) op_driver.cc:348] REPLICATION: starting
0709 12:13:42.799009 (+ 40283us) log.cc:844] Serialized 533 byte log entry
0709 12:13:42.876790 (+ 77781us) op_driver.cc:464] REPLICATION: finished
0709 12:13:43.974992 (+1098202us) write_op.cc:301] APPLY: starting
0709 12:13:43.975089 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:43.975237 (+   148us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:44.073664 (+ 98427us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:44.088621 (+ 14957us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:44.130510 (+ 41889us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:44.141614 (+ 11104us) write_op.cc:312] APPLY: finished
0709 12:13:44.176168 (+ 34554us) log.cc:844] Serialized 23 byte log entry
0709 12:13:45.203865 (+1027697us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:45.203957 (+    92us) write_op.cc:454] Released schema lock
0709 12:13:45.203994 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:45.204998 (+  1004us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:45.205027 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23352,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":1058910,"apply.run_cpu_time_us":58607,"apply.run_wall_time_us":1229426,"mutex_wait_us":1139,"num_ops":1,"prepare.queue_time_us":45328,"prepare.run_cpu_time_us":6107,"prepare.run_wall_time_us":238260,"raft.queue_time_us":1372384,"raft.run_cpu_time_us":129880,"raft.run_wall_time_us":2166027,"replication_time_us":117898,"thread_start_us":2448949,"threads_started":5,"wal-append.queue_time_us":823634,"wal-append.run_cpu_time_us":9056,"wal-append.run_wall_time_us":1065946}]]}
W20260709 12:13:45.539768 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.547624 25835 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:45.559638 25585 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:45.564345 25856 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.565089 25907 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.565745 25855 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.566672 25871 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.572274 25832 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.557185 25912 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.592531 25836 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:45.570123 25853 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.605897 25929 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.608517 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.609777 23932 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 1.
I20260709 12:13:45.540047 23683 rpcz_store.cc:276] 0709 12:13:42.443492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.443684 (+   192us) service_pool.cc:224] Handling call
0709 12:13:45.539390 (+3095706us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:42.835465 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:42.835997 (+   532us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:42.836025 (+    28us) write_op.cc:435] Acquired schema lock
0709 12:13:42.836037 (+    12us) tablet.cc:662] Decoding operations
0709 12:13:42.836376 (+   339us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:42.836427 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:42.836463 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:42.836651 (+   188us) tablet.cc:701] Row locks acquired
0709 12:13:42.836665 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:13:42.836795 (+   130us) write_op.cc:270] Start()
0709 12:13:42.836898 (+   103us) write_op.cc:276] Timestamp: P: 1783599222836750 usec, L: 0
0709 12:13:42.836916 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:13:42.840070 (+  3154us) log.cc:844] Serialized 538 byte log entry
0709 12:13:44.224405 (+1384335us) op_driver.cc:464] REPLICATION: finished
0709 12:13:45.206461 (+982056us) write_op.cc:301] APPLY: starting
0709 12:13:45.206541 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:45.206645 (+   104us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:45.247794 (+ 41149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:45.248062 (+   268us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:45.248241 (+   179us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:45.248291 (+    50us) write_op.cc:312] APPLY: finished
0709 12:13:45.248480 (+   189us) log.cc:844] Serialized 23 byte log entry
0709 12:13:45.248650 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:45.248734 (+    84us) write_op.cc:454] Released schema lock
0709 12:13:45.248772 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:45.249126 (+   354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:45.249149 (+    23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75296,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":981751,"apply.run_cpu_time_us":1852,"apply.run_wall_time_us":42657,"num_ops":1,"prepare.queue_time_us":239613,"prepare.run_cpu_time_us":2427,"prepare.run_wall_time_us":4911,"replication_time_us":1387284,"thread_start_us":831064,"threads_started":1}]]}
I20260709 12:13:45.613350 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:45.615670 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.009s	sys 0.022s
W20260709 12:13:45.626338 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.010s	sys 0.023s
W20260709 12:13:45.638195 23781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.005s	sys 0.034s
I20260709 12:13:45.643097 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 266) took 1662 ms. Trace:
I20260709 12:13:45.643755 23680 rpcz_store.cc:276] 0709 12:13:43.970589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.979708 (+  9119us) service_pool.cc:224] Handling call
0709 12:13:44.227490 (+247782us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:45.633000 (+1405510us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6,"spinlock_wait_cycles":341865856}
W20260709 12:13:45.654081 25851 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.696s	user 0.016s	sys 0.028s
W20260709 12:13:45.654852 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.654711 25851 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:45.658252 23781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.006s	sys 0.034s
W20260709 12:13:45.672458 23784 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.833s	user 0.023s	sys 0.027s
I20260709 12:13:45.672999 23784 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:45.674798 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:45.676293 23682 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 273) took 2916 ms. Trace:
I20260709 12:13:45.676867 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:45.676712 23682 rpcz_store.cc:276] 0709 12:13:42.759352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.761059 (+  1707us) service_pool.cc:224] Handling call
0709 12:13:42.846204 (+ 85145us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:45.676100 (+2829896us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2522,"reported_tablets":2,"spinlock_wait_cycles":177005824}
I20260709 12:13:45.680030 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 95) took 2269 ms. Trace:
I20260709 12:13:45.681972 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:45.679145 25846 consensus_meta.cc:287] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.604s	user 0.021s	sys 0.020s
W20260709 12:13:45.697263 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 262867us
I20260709 12:13:45.698698 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:45.681416 25854 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:45.682073 23853 rpcz_store.cc:276] 0709 12:13:43.410154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.463163 (+ 53009us) service_pool.cc:224] Handling call
0709 12:13:43.463526 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.437688 (+974162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.437718 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.438219 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.142270 (+704051us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:45.581881 (+439611us) log.cc:844] Serialized 14 byte log entry
0709 12:13:45.647791 (+ 65910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.679529 (+ 31738us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.679576 (+    47us) raft_consensus.cc:1685] finished
0709 12:13:45.679606 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.679831 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47729,"prepare.queue_time_us":439893,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":3033,"raft.queue_time_us":79361,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":10356,"spinlock_wait_cycles":22493440,"thread_start_us":2021403,"threads_started":3,"wal-append.queue_time_us":812399}
I20260709 12:13:45.709313 25176 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:45.722329 25844 consensus_meta.cc:287] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.014s
I20260709 12:13:45.723779 25845 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.724516 25845 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:45.732492 25861 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.780s	user 0.021s	sys 0.027s
I20260709 12:13:45.733106 25861 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.734325 25941 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:45.722900 25844 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.735102 25941 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.700906 25846 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:45.737481 25844 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.738854 25846 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:45.741890 23779 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:45.743606 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:45.746258 25796 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.732960 25906 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.747678 25172 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:45.755297 25923 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:13:45.747570 25901 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.015s
I20260709 12:13:45.756022 25901 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:45.758183 25870 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.822s	user 0.013s	sys 0.010s
I20260709 12:13:45.752394 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:45.753041 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:45.748169 25906 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:45.763023 25895 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.763293 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:45.763741 25895 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.766371 25895 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.766937 25844 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.767829 25844 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.758847 25870 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.770010 23858 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:45.770488 23853 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:45.770793 25844 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.771359 25844 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:45.772053 25844 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.751950 25929 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:45.773227 25929 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:45.774336 25844 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.772444 25936 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.775749 25906 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:45.776391 25845 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.759747 23854 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:45.753167 23851 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:45.753628 23776 consensus_meta.cc:287] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.812s	user 0.013s	sys 0.000s
I20260709 12:13:45.758150 25901 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:45.800810 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 83385.4us
I20260709 12:13:45.777041 25906 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.749342 23931 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:45.812758 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:45.749965 25858 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.611s	user 0.020s	sys 0.020s
I20260709 12:13:45.759714 25925 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:45.748498 25851 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.751114 25861 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.775859 25936 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.737622 25941 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:45.814129 23854 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:45.813990 23776 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:45.824987 23898 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:45.754413 25885 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:45.826228 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.812083 25929 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.831882 25858 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.823462 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.843408 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:45.844228 25891 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.849s	user 0.002s	sys 0.024s
I20260709 12:13:45.754844 25669 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.867059 25891 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:45.867434 25845 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:13:45.868296 25843 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:13:45.827533 25925 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.872879 23925 raft_consensus.cc:2410] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:45.876708 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:45.820322 25870 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.881489 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 49) took 1112 ms. Trace:
I20260709 12:13:45.882032 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.883057 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:45.825167 25849 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.010s
I20260709 12:13:45.887933 25864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e60a1c6520545729d27474d8f237ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:45.825263 25837 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:45.904136 23776 rpcz_store.cc:276] 0709 12:13:44.768647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.768772 (+   125us) service_pool.cc:224] Handling call
0709 12:13:45.881238 (+1112466us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54000a0854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:45.881403 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248991488}
I20260709 12:13:45.910614 25855 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:45.864774 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 1996) took 1754 ms (client timeout 1633 ms). Trace:
I20260709 12:13:45.914170 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:45.909562 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 88052.3us
W20260709 12:13:45.916705 23931 rpcz_store.cc:269] 0709 12:13:44.080539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.080772 (+   233us) service_pool.cc:224] Handling call
0709 12:13:45.274461 (+1193689us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b540007d8a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad729dc0 00007fb2ad7b5e18 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:45.830197 (+555736us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916590848}
I20260709 12:13:45.934150 23934 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:45.935539 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.938097 25917 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:45.939292 25837 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:45.941804 25837 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:45.920112 25855 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.825062 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:45.916529 25796 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:45.946846 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 168513us
I20260709 12:13:45.948257 25917 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:45.948748 25929 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:45.949879 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:46.036087 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.107335 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.053689 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:45.952634 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:46.081858 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.115478 23853 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:46.049810 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.034475 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.148558 23858 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:46.112805 25917 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:13:46.152480 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 202704us
I20260709 12:13:46.039253 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:45.990814 25845 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:46.034224 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.157888 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:46.039877 25925 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:46.161409 25918 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:46.162163 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:46.162716 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:46.164948 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.166440 23934 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.171793 25835 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:46.181181 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.174497 25858 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:46.175210 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:46.175241 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.175882 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.176399 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.177752 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.178743 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.179025 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.180068 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.180348 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.181126 25917 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:46.182317 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:46.182545 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 357640us
I20260709 12:13:46.182785 25796 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:45.990667 25669 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:46.060307 25943 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:46.061363 25901 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:46.114185 25929 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:46.117560 25855 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.186784 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.040714 25843 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:46.057868 25849 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.061313 25861 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:45.954493 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.193940 23925 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.081377 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet de3c70defd864769b26b52b7f36548cd from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:13:46.212229 23927 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.191112 25918 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:46.230088 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.233494 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.236194 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.198307 25854 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:46.201901 25796 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:46.239872 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.236925 23852 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:46.244933 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:46.246071 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.243333 25796 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.249217 23852 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:46.250547 23852 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:46.248639 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.284806 23933 raft_consensus.cc:2393] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:13:46.229859 25901 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.289947 25901 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:46.291831 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:46.292637 23926 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.293243 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.294667 25843 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.237931 25918 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.295425 25885 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:40118 (request call id 3) took 5040 ms. Trace:
I20260709 12:13:46.208801 25845 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.241815 25835 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.218264 25891 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:46.315696 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:46.253523 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet de3c70defd864769b26b52b7f36548cd from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-de3c70defd864769b26b52b7f36548cd
I20260709 12:13:46.237393 25943 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.243369 25849 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.478649 23934 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:46.820693 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.486424 23927 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:46.726399 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.816604 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:46.725894 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:46.801141 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:46.361289 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 170878us
I20260709 12:13:46.698045 23925 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:46.780051 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:46.739704 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.020s	sys 0.022s
W20260709 12:13:46.854446 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.020s	sys 0.030s
I20260709 12:13:46.726999 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:46.574282 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:46.852623 23852 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:46.511703 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 314937us
I20260709 12:13:46.780653 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:46.684653 25952 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 12:13:46.611263 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:46.841202 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 196519us
W20260709 12:13:46.812362 25855 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.000s
I20260709 12:13:46.853892 23933 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:46.321794 25835 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:46.816926 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:46.777674 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.697355 25917 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.433149 25936 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:46.776448 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.865384 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:46.867352 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 140459us
I20260709 12:13:46.319180 25845 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:46.869510 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:46.814802 25858 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.667565 25891 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:46.730086 25843 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:46.315925 25885 rpcz_store.cc:276] 0709 12:13:41.254830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.254992 (+   162us) service_pool.cc:224] Handling call
0709 12:13:41.284781 (+ 29789us) log.cc:844] Serialized 499 byte log entry
0709 12:13:46.295291 (+5010510us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":706129,"threads_started":1,"wal-append.queue_time_us":71638,"wal-append.run_cpu_time_us":1636,"wal-append.run_wall_time_us":1007533}
I20260709 12:13:46.878934 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:46.879665 23778 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.883809 23781 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:46.884027 23929 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:46.884181 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.812790 25829 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:46.565896 23682 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:46.873155 23779 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.301952 25861 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.574831 25894 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:46.731010 25925 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:46.871417 23777 raft_consensus.cc:2376] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5. Re-sending same reply.
I20260709 12:13:46.814810 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.885004 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:46.885718 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.887053 23926 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:46.316067 25854 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.815150 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.892134 23852 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:46.890131 25836 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:46.552069 25796 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.896085 23778 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:46.891162 23682 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:46.912979 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:46.913192 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.913805 23929 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:46.888787 25918 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.571s	user 0.005s	sys 0.005s
I20260709 12:13:46.901202 23932 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:46.668048 25929 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.894742 25829 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:46.930668 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.870035 25855 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:46.896656 25894 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:46.939105 25176 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:46.937348 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 75392.3us
I20260709 12:13:46.935643 25174 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:46.938390 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.888526 25858 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.940312 25119 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:46.946262 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.909727 25836 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:46.906952 23855 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:46.950726 23900 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:46.953590 25929 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.959728 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:46.912494 25838 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:46.979151 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 63773.5us
I20260709 12:13:46.944087 25894 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.972201 25836 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:46.982363 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 217404us
I20260709 12:13:46.937041 25170 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:46.931748 25891 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.990288 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:46.931290 25918 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:46.943709 25925 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.004453 25958 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:47.004987 25958 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:47.005311 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.005496 25958 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.007006 25835 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:47.007426 25835 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:47.007797 25835 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:47.008212 25862 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:47.009145 25829 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.008864 25862 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:47.010227 25796 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:47.005153 25953 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:13:47.011312 25862 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:46.993707 25936 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:47.037156 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:47.042925 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 91302.5us
W20260709 12:13:47.048545 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45906.3us
W20260709 12:13:47.051652 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 78536us
I20260709 12:13:47.002609 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:47.054924 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:47.053512 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:46.992486 25838 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:47.109261 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55826.2us
I20260709 12:13:47.110101 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:47.037180 25936 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:47.126196 25970 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:47.126965 25970 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.128185 25796 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:47.128616 25796 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:47.128975 25796 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:47.129423 25970 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.129724 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:47.052158 25862 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:47.137964 23779 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:47.100633 25849 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.129463 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:47.175531 23859 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:47.121429 25936 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.179723 25971 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:47.179885 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:47.180851 25925 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:47.185083 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.189219 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 115) took 1020 ms. Trace:
W20260709 12:13:47.189620 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 134524us
I20260709 12:13:47.189648 23854 rpcz_store.cc:276] 0709 12:13:46.168296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.173361 (+  5065us) service_pool.cc:224] Handling call
0709 12:13:46.173716 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.174132 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:46.396581 (+222449us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.188568 (+791987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:47.188626 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:47.188673 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.188744 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.189154 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56617,"prepare.queue_time_us":640906,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":122805,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":642895,"threads_started":3,"wal-append.queue_time_us":403137}
I20260709 12:13:47.170007 25862 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:47.197746 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:47.186760 25962 ts_tablet_manager.cc:933] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:13:47.208242 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 96658.3us
I20260709 12:13:47.174460 25169 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:47.209702 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:47.209872 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 99861us
I20260709 12:13:47.212940 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:47.201378 25943 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.200613 25854 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:47.205716 25862 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.216061 23928 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:47.214004 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:47.210511 25957 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:47.238106 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:47.216321 25855 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.243393 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:47.237890 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5671595139999681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 12:13:47.245472 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:47.247022 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.243410 25943 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:47.247967 25961 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:13:47.248031 25943 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:47.212345 25966 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:47.246693 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0765466180000658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
I20260709 12:13:47.208632 25971 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.215494 25906 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:47.252327 25979 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:47.258239 25979 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:47.258632 25979 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.262636 23856 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:47.238020 25925 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:47.248571 25943 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:47.264705 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:47.243412 25854 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:47.243691 25829 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:47.273746 25959 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.287s	user 0.002s	sys 0.002s
W20260709 12:13:47.291347 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.254825 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:47.322714 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 812) took 1071 ms. Trace:
I20260709 12:13:47.332785 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.277539 25891 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.338795 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 810) took 1088 ms. Trace:
I20260709 12:13:47.339214 23856 rpcz_store.cc:276] 0709 12:13:46.250466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.858996 (+608530us) service_pool.cc:224] Handling call
0709 12:13:47.338630 (+479634us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002f67d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:47.338734 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236762496}
I20260709 12:13:47.328011 25829 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.326263 25977 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:47.347196 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.586s (SENT)
I20260709 12:13:47.331928 25170 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:47.390992 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 138103us
I20260709 12:13:47.362538 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 811) took 1111 ms. Trace:
I20260709 12:13:47.330832 23855 rpcz_store.cc:276] 0709 12:13:46.251101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.859655 (+608554us) service_pool.cc:224] Handling call
0709 12:13:46.876289 (+ 16634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.885612 (+  9323us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:47.026096 (+140484us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.322312 (+296216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:47.322374 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:47.322419 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.322489 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.322620 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13986,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":46170,"thread_start_us":254120,"threads_started":1}
W20260709 12:13:47.393796 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 129269us
I20260709 12:13:47.393918 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:47.363373 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 116) took 1194 ms. Trace:
I20260709 12:13:47.343916 25977 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.401667 25967 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:47.391043 25838 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:47.390359 25974 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:13:47.389495 25864 consensus_meta.cc:287] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.000s
I20260709 12:13:47.326967 25925 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:47.401153 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 167054us
I20260709 12:13:47.398108 23858 rpcz_store.cc:276] 0709 12:13:46.168608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.184866 (+ 16258us) service_pool.cc:224] Handling call
0709 12:13:46.195556 (+ 10690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.347457 (+151901us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:46.863891 (+516434us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.323188 (+459297us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:47.323234 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:47.323274 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.363122 (+ 39848us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540037f854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:47.363172 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.363313 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":416295,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":134316,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":302483328,"thread_start_us":785449,"threads_started":3,"wal-append.queue_time_us":322912}
I20260709 12:13:47.397105 23860 rpcz_store.cc:276] 0709 12:13:46.250804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.860146 (+609342us) service_pool.cc:224] Handling call
0709 12:13:46.868830 (+  8684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.922564 (+ 53734us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:46.951371 (+ 28807us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.217435 (+266064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:47.235651 (+ 18216us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:47.235751 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.345201 (+109450us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54000a14d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:47.345242 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.362461 (+ 17219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57888,"prepare.queue_time_us":43283,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":997,"raft.queue_time_us":69949,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":9102,"spinlock_wait_cycles":98174336,"thread_start_us":173383,"threads_started":2,"wal-append.queue_time_us":142933}
W20260709 12:13:47.421308 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74159.4us
I20260709 12:13:47.421849 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:47.326251 25906 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.420071 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:47.394663 25918 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:47.398921 25854 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:47.407192 25917 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.000s
I20260709 12:13:47.395728 25858 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.444686 25835 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.445178 25864 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44500 (request call id 335) took 4793 ms. Trace:
I20260709 12:13:47.318841 25966 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.407191 25862 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.389086 25796 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.424062 25973 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:47.447583 25829 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.382498 25979 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.479534 25838 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:47.481786 25176 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:47.482888 25796 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.418352 25894 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:47.518014 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98087.3us
I20260709 12:13:47.519913 25986 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:47.520407 25986 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:47.520794 25986 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:47.519222 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:47.552421 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.453707 25836 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.626387 23779 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:47.679347 25966 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.593885 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.680298 23782 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:47.681309 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.682500 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.683781 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.584249 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:47.620846 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.684340 23932 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:47.686163 23780 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:47.686635 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.687341 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.688534 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.691138 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.691802 23779 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:47.692651 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.693316 23779 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
W20260709 12:13:47.693650 25958 consensus_meta.cc:287] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.003s
I20260709 12:13:47.694082 25958 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:47.695621 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.696285 25977 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.485445 25988 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.194s
I20260709 12:13:47.696612 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:47.697295 25906 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.697762 25982 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:47.698490 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.700618 23779 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.620371 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.701821 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.618782 25973 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:47.786867 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:47.598600 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 79493us
W20260709 12:13:47.984200 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 134903us
I20260709 12:13:47.811080 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.985948 23776 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:47.872020 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:47.985728 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 85072.9us
I20260709 12:13:47.791448 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:47.712536 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:47.983719 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65623.9us
I20260709 12:13:47.984514 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:47.844252 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:47.743908 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.667702 25854 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.807801 25958 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:47.739274 25894 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.996445 23857 raft_consensus.cc:2410] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:13:47.622495 25925 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:47.776532 25838 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:47.994158 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 224466us
I20260709 12:13:47.623200 25971 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:48.045580 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.052415 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.101406 25838 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:47.453504 25864 rpcz_store.cc:276] 0709 12:13:42.651960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.652127 (+   167us) service_pool.cc:224] Handling call
0709 12:13:42.893070 (+240943us) ts_tablet_manager.cc:934] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
0709 12:13:43.926201 (+1033131us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4c00116940. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aba7ea20 00007fb2aba8fa29 00007fb2aba9013f 00007fb2aba542ab 00007fb2aba81f3c 00007fb2aba82461 00007fb2aba82768 00007fb2b18082eb 00007fb2b272dd3b 00007fb2b2728008 00007fb2b2727405
0709 12:13:45.886021 (+1959820us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:46.767866 (+881845us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:46.817674 (+ 49808us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:47.442998 (+625324us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:47.444534 (+  1536us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:47.444873 (+   339us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:47.445084 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":60307,"spinlock_wait_cycles":5138688,"tablet-copy.queue_time_us":188372,"thread_start_us":759926,"threads_started":2}
I20260709 12:13:48.103827 25864 tablet_copy_client.cc:806] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:48.104761 25864 tablet_copy_client.cc:670] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:48.094352 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:47.804150 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 72964.4us
I20260709 12:13:48.106760 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:47.983474 25979 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.953636 25836 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:47.742928 25936 consensus_meta.cc:287] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.005s
W20260709 12:13:48.100272 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.633s (SENT)
I20260709 12:13:48.008611 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.007826 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:48.108469 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.027729 23932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:48.109082 23932 raft_consensus.cc:606] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:48.109365 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.777s (SENT)
I20260709 12:13:47.590961 25861 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.740160 25835 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.109930 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.110458 25936 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:48.111104 23825 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.793s (SENT)
I20260709 12:13:48.109951 23682 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 163) took 2001 ms. Trace:
I20260709 12:13:47.744064 25943 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.107569 25979 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:48.111016 23680 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:48.112897 25979 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.111701 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:48.114099 25979 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.114703 25858 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:48.113991 23680 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 5, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:47.577342 25957 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:48.116071 23680 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 3 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:48.116581 25957 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:48.116618 23825 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.842s (SENT)
I20260709 12:13:48.111534 25861 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.117797 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.119128 25173 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.119283 25858 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:47.782895 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.120527 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.874s (SENT)
I20260709 12:13:48.121026 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.111701 23682 rpcz_store.cc:276] 0709 12:13:46.161848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.162892 (+  1044us) service_pool.cc:224] Handling call
0709 12:13:48.109891 (+1946999us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:47.122191 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:47.122835 (+   644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:47.122899 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:13:47.122919 (+    20us) tablet.cc:662] Decoding operations
0709 12:13:47.123528 (+   609us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:47.123630 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:47.123663 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:47.124169 (+   506us) tablet.cc:701] Row locks acquired
0709 12:13:47.124215 (+    46us) write_op.cc:260] PREPARE: finished
0709 12:13:47.124402 (+   187us) write_op.cc:270] Start()
0709 12:13:47.124538 (+   136us) write_op.cc:276] Timestamp: P: 1783599227124324 usec, L: 0
0709 12:13:47.124560 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:13:47.125388 (+   828us) log.cc:844] Serialized 913 byte log entry
0709 12:13:47.849478 (+724090us) op_driver.cc:464] REPLICATION: finished
0709 12:13:48.096783 (+247305us) write_op.cc:301] APPLY: starting
0709 12:13:48.096881 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:48.097079 (+   198us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:48.097323 (+   244us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:48.097770 (+   447us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:48.098011 (+   241us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:13:48.098054 (+    43us) write_op.cc:312] APPLY: finished
0709 12:13:48.098349 (+   295us) log.cc:844] Serialized 29 byte log entry
0709 12:13:48.098652 (+   303us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:48.098742 (+    90us) write_op.cc:454] Released schema lock
0709 12:13:48.098795 (+    53us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":227205,"num_ops":2,"prepare.queue_time_us":166467,"prepare.run_cpu_time_us":54205,"prepare.run_wall_time_us":89352,"raft.queue_time_us":384215,"raft.run_cpu_time_us":7824,"raft.run_wall_time_us":205719,"replication_time_us":668491,"thread_start_us":296874,"threads_started":4,"wal-append.queue_time_us":92181}]]}
I20260709 12:13:48.112042 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:48.122900 25943 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:48.048219 23900 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.123245 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.123890 25835 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.124639 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.742295 23870 tablet_copy_source_session.cc:215] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:13:47.963351 25170 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.000s
W20260709 12:13:47.935264 25929 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.959s	user 0.006s	sys 0.005s
I20260709 12:13:47.792572 25987 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.112408 25971 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:48.125025 25861 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:48.126411 25849 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.126940 25849 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.126137 23783 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:48.129230 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.129840 23783 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:47.839500 25962 tablet_copy_client.cc:323] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:13:48.125716 25936 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:48.130754 25849 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:48.038033 23899 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.001s	sys 0.005s
W20260709 12:13:48.131667 23899 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.003s	sys 0.005s
W20260709 12:13:48.131984 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 688020us
I20260709 12:13:48.133179 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet d6d3d4f634284532bbf6ad7d4ac19214 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:13:48.133431 25835 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.133797 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet d6d3d4f634284532bbf6ad7d4ac19214 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-d6d3d4f634284532bbf6ad7d4ac19214
I20260709 12:13:48.134214 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.133960 25835 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.134857 23781 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:48.135813 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.136322 25835 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:48.136350 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:48.137008 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:48.137639 25936 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:48.138015 25936 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.138535 25936 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.140136 25958 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.140524 25982 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.140720 25958 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.141028 25982 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.142827 25958 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.142896 25982 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.143994 25982 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.144222 25113 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.144487 25982 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.145020 25894 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.146584 25982 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.031997 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:48.138046 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:48.148121 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.148763 23781 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:48.027145 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:48.152706 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 165322us
I20260709 12:13:48.154268 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:47.939493 25986 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.126155 25971 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.156114 23854 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:48.156694 25971 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.159983 25986 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:47.792325 25855 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:48.155066 25936 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.162042 25855 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:48.163293 23870 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:46448 (request call id 111) took 2149 ms. Trace:
I20260709 12:13:48.163113 25855 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.163729 23870 rpcz_store.cc:276] 0709 12:13:46.066126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.079034 (+ 12908us) service_pool.cc:224] Handling call
0709 12:13:48.163203 (+2084169us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:48.161685 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.165248 25835 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:48.166334 25835 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:48.166678 25835 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:48.166887 23776 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:48.167258 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.167572 25835 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.168126 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.168809 25835 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.170578 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.171790 25987 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.172827 25958 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.173383 25958 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.173684 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.174041 25987 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:48.174209 23929 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.108444 23782 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:48.177765 25958 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.179239 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:48.124774 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76682.2us
I20260709 12:13:48.182390 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:48.183622 23929 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:48.184090 25958 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:48.007721 25872 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.184520 25958 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.184938 25958 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.184917 25872 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.185544 23776 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:13:48.185139 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62942.5us
I20260709 12:13:48.186993 25987 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:48.187636 25872 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:48.161036 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:47.535061 25862 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.188344 25973 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.189965 25936 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.192188 25887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de3c70defd864769b26b52b7f36548cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:48.197587 25958 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.205910 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.211122 23779 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:48.213027 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:48.265569 23777 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:48.215685 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.266130 23783 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:48.205945 23795 tablet_copy_source_session.cc:215] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:48.164624 25855 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:48.217255 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 63067.8us
I20260709 12:13:48.267943 25861 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.268110 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.218046 25925 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:48.268661 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 156715us
I20260709 12:13:48.364810 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:48.334517 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:48.367748 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 161939us
I20260709 12:13:48.368680 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:48.373292 23776 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.125597 25170 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
W20260709 12:13:48.369122 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 101077us
I20260709 12:13:48.375480 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.373842 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:48.261823 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 46237.3us
I20260709 12:13:48.379746 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.380381 23824 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:48.384864 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.386854 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.270524 25906 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:48.389602 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.339401 25958 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.269598 25872 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:13:48.329700 25862 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.393078 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.393786 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:48.467314 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.386168 25861 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.465687 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.472322 23853 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.466784 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:48.395941 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.474138 23854 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:48.453915 23933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:48.474931 23933 raft_consensus.cc:606] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:48.445382 25176 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.475544 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:48.476148 23899 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.601s (SENT)
I20260709 12:13:48.476576 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.379570 25979 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:48.432075 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:48.485567 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.486222 23926 raft_consensus.cc:2376] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3. Re-sending same reply.
I20260709 12:13:48.505594 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.506304 23934 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.390669 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 553) took 1166 ms. Trace:
I20260709 12:13:48.509594 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:48.510854 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.511453 23926 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.511893 23928 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
W20260709 12:13:48.512655 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 80714.5us
I20260709 12:13:48.392532 26001 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:48.404866 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.337100 25973 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.514814 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.533730 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.534430 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.517761 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.536260 23925 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:48.487519 25979 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:48.540938 23857 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:48.519387 25987 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:48.524596 25169 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:48.543232 25962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6d3d4f634284532bbf6ad7d4ac19214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:48.525511 25864 tablet_copy_client.cc:538] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:13:48.548920 26004 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:13:48.550006 25987 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:48.557168 23859 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:48.551196 25987 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.559887 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.560472 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:48.514043 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:48.532998 23853 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:13:48.522877 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46761.8us
I20260709 12:13:48.541870 25796 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:48.542272 25838 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:48.516084 25835 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:48.571256 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 184340us
I20260709 12:13:48.572132 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:48.575717 23776 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:48.577978 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.424474 25971 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:48.589257 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:48.563936 25979 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.582548 25796 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:48.564417 23930 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:48.584589 25994 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:48.574014 25838 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.529011 26001 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:48.586186 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.587921 23934 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:48.590286 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:48.590693 23854 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.581238 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:48.513427 25170 rpcz_store.cc:276] 0709 12:13:47.217663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.233167 (+ 15504us) service_pool.cc:224] Handling call
0709 12:13:48.381069 (+1147902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.528815 25872 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:13:48.584967 25966 raft_consensus.cc:2764] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:13:48.602483 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.587179 25906 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:48.603729 23925 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:13:48.611406 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.612941 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.616852 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:48.592535 25977 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.595335 25835 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.601195 25838 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.632627 23926 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:48.594142 25971 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.658581 26011 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:48.659173 26011 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:48.659543 26011 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.665064 26014 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:48.634486 23825 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.647s (SENT)
I20260709 12:13:48.586944 25967 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:13:48.670499 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.672019 23855 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:48.672502 23855 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:48.600736 26001 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.612221 25894 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:13:48.677809 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1722616950000884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:13:48.679040 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0678616949999196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 12:13:48.679723 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.682152 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.673362 23855 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.614640 25906 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:48.682845 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 79478.4us
W20260709 12:13:48.688422 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 107261us
I20260709 12:13:48.627167 25982 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.638669 25978 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:48.691330 25894 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:48.692822 23898 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:48.613801 25925 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
W20260709 12:13:48.700374 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.700455 23855 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.690838 25906 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.704520 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:48.724098 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.726222 23776 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.699227 25849 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.638774 25977 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:48.683918 25966 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:13:48.692718 25994 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:48.656178 25835 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:48.733392 23779 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:48.744951 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.734064 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.694262 25957 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:48.699858 25978 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:48.706566 25958 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:13:48.739399 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.753248 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.740790 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.743835 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.743250 23855 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:48.706056 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.730437 25862 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.755288 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.730688 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.765126 23934 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:48.772444 23782 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:48.765457 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.765588 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.765805 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.756322 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.699401 25796 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.776707 23925 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:48.766736 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:48.772583 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.783037 23934 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:13:48.777786 23825 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.914s (SENT)
I20260709 12:13:48.779455 25872 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:13:48.780011 26007 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:13:48.781579 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:48.788352 23933 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:48.790171 23782 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.751713 25957 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:48.794426 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.796365 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.797770 23931 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:48.768318 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:48.764055 25986 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.768860 25864 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:48.769047 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:48.835790 23781 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.781518 25862 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:48.837952 26009 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:13:48.782413 25958 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:48.784309 25872 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:48.793246 25925 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:48.868886 25977 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.804745 25957 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.868547 25872 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.755831 25971 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:48.869831 25925 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:48.909783 25843 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:48.886337 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 181874us
I20260709 12:13:48.934571 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:48.919378 23930 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:48.964721 26011 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.965982 23776 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:48.919723 26006 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 12:13:48.954295 25925 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.905663 25978 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.956403 25979 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:48.955497 25849 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.960258 25872 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.920723 25843 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:48.953544 25855 consensus_meta.cc:287] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.009s
I20260709 12:13:48.831638 25894 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:48.976071 23899 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.646s (SENT)
I20260709 12:13:48.977597 23782 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.007972 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:48.918092 25994 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:48.977942 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:49.014123 23783 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:48.978727 23825 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.923s (SENT)
I20260709 12:13:49.009215 25977 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:48.982069 26023 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.143s	user 0.010s	sys 0.018s
I20260709 12:13:49.049738 23854 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:49.085845 23932 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:13:49.087532 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.116242 25835 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:49.117753 26001 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:49.119372 25838 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.125079 25862 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:48.990947 25958 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:49.161813 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.174634 23783 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:49.209020 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 104) took 1040 ms. Trace:
I20260709 12:13:49.209496 23932 rpcz_store.cc:276] 0709 12:13:48.168345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.170336 (+  1991us) service_pool.cc:224] Handling call
0709 12:13:49.208801 (+1038465us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54002a3d54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:49.208944 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916737920}
I20260709 12:13:49.233604 25906 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.227084 25957 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:49.285427 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.014s	sys 0.002s
W20260709 12:13:49.457702 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.014s	sys 0.012s
I20260709 12:13:49.458398 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.458786 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:49.281060 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.460026 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 481285us
W20260709 12:13:49.451357 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.461743 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.451685 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 517136us
W20260709 12:13:49.250077 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.451339 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.465878 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.466481 23825 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.970s (SENT)
W20260709 12:13:49.269080 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.010s	sys 0.005s
W20260709 12:13:49.468155 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.017s	sys 0.009s
I20260709 12:13:49.451455 25925 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:49.460834 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.474833 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:49.468562 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 746540us
I20260709 12:13:49.236497 26001 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:49.489159 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.451536 25887 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:44500 (request call id 338) took 6401 ms. Trace:
I20260709 12:13:49.305675 25858 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:49.492955 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.912s (SENT) [suppressed 62 similar messages]
I20260709 12:13:49.347807 25906 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:49.499140 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.499526 25843 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:49.500160 26011 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:49.486096 25973 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:13:49.526877 23859 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:49.526818 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.502367 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.527976 25176 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.533103 26001 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:49.513338 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.533530 26001 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:49.534545 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.532913 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.533157 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.534399 26001 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.535928 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.536669 23860 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:49.550266 23856 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:49.551558 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.552749 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.552832 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.559202 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.559966 23778 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:49.560994 23853 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:49.563019 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 102782us
I20260709 12:13:49.568333 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:49.569051 23854 raft_consensus.cc:2376] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4. Re-sending same reply.
W20260709 12:13:49.570055 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.570348 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.553164 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:49.571695 23858 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:49.574522 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.525538 25979 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:49.523867 25838 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:49.580302 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.581472 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.580478 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.576907 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:49.583269 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.584059 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:49.589354 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 96437.4us
I20260709 12:13:49.588210 23779 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:49.547986 25843 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.624464 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:49.592404 23783 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:49.625242 23776 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:49.626154 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:49.568200 25894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.036s	sys 0.034s
W20260709 12:13:49.629999 25894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.036s	sys 0.034s
I20260709 12:13:49.624200 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:49.633126 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:49.500640 26025 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:13:49.627979 23856 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:49.629029 25843 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.624760 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:49.633859 23825 leader_election.cc:336] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.588s (SENT)
W20260709 12:13:49.640462 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.619s (SENT)
I20260709 12:13:49.642791 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:49.547789 25872 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 12:13:49.645052 23860 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:49.635015 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46135.7us
I20260709 12:13:49.648056 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:49.649354 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:49.650466 25973 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:49.650570 25843 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:49.651087 25971 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:49.652307 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.585597 26027 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:49.658018 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:49.642040 25979 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:49.658638 25978 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.044s	sys 0.064s
W20260709 12:13:49.698520 25978 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.044s	sys 0.064s
I20260709 12:13:49.666643 26018 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:49.661518 26020 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:49.668401 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.700618 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.655057 25973 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:49.662249 26001 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.750474 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:49.751571 23928 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:49.738205 23751 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.638s (SENT)
I20260709 12:13:49.704815 26018 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:49.850987 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.765902 23777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:53932
I20260709 12:13:49.886906 23777 raft_consensus.cc:606] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:49.887735 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 320485us
W20260709 12:13:49.886168 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.891533 23928 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
W20260709 12:13:49.846163 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.724800 26027 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:49.893801 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.724057 25973 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:49.896668 23776 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:49.491983 25887 rpcz_store.cc:276] 0709 12:13:43.050286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.050403 (+   117us) service_pool.cc:224] Handling call
0709 12:13:44.076632 (+1026229us) ts_tablet_manager.cc:934] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
0709 12:13:48.190771 (+4114139us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:48.990124 (+799353us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:49.021289 (+ 31165us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:49.354930 (+333641us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:49.411084 (+ 56154us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:49.451344 (+ 40260us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:49.451469 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":1989,"tablet-copy.queue_time_us":920463,"thread_start_us":1716375,"threads_started":2}
I20260709 12:13:49.726915 25971 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:49.729187 26020 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:49.916525 23825 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.714s (SENT)
I20260709 12:13:49.847007 25894 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:49.919883 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 181712us
I20260709 12:13:49.921018 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:49.855468 25962 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53494 (request call id 137) took 3036 ms. Trace:
W20260709 12:13:49.916756 26013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.020s	sys 0.025s
W20260709 12:13:49.933357 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 275416us
I20260709 12:13:49.935386 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:49.918958 25887 tablet_copy_client.cc:806] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
W20260709 12:13:49.939074 26037 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:13:49.930891 25894 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.941437 25894 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:49.942456 23825 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.820s (SENT)
I20260709 12:13:49.942692 25979 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:49.919283 25971 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:49.914227 25936 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:13:49.928328 25173 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.992106 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:50.001320 26049 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:49.996798 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.001631 25925 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.000s
I20260709 12:13:49.991886 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:49.985651 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.021579 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.023928 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.024518 23778 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:50.021785 23782 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:50.019388 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:50.027168 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:49.984916 25887 tablet_copy_client.cc:670] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:50.028563 25979 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.029039 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.028873 26027 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.029361 25979 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.041731 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.028662 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:49.935029 25169 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:49.996181 26034 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 12:13:50.072930 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.028398 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:49.934705 26013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.020s	sys 0.025s
W20260709 12:13:50.072868 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 130437us
I20260709 12:13:50.046495 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.074069 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.076354 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.906s (SENT)
I20260709 12:13:50.075927 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.075927 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:50.076951 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 155327us
W20260709 12:13:50.077191 23825 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.907s (SENT)
W20260709 12:13:50.078181 23825 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.055s (SENT)
I20260709 12:13:50.001899 26049 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:13:50.078609 23780 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:50.076866 23778 raft_consensus.cc:2376] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:50.080657 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.082599 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.083801 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.084745 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.086268 23784 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:50.083268 23785 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:50.100648 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.102624 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.104351 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.108484 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.109073 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 159469us
I20260709 12:13:50.113122 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.114370 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.082346 26049 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:50.114805 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.082175 25843 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:50.112278 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:50.124500 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.124696 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.074s (SENT)
I20260709 12:13:50.001672 25971 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.101413 26050 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.129714 26050 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.131163 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2041) took 1044 ms. Trace:
I20260709 12:13:50.131577 23782 rpcz_store.cc:276] 0709 12:13:49.087103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.087237 (+   134us) service_pool.cc:224] Handling call
0709 12:13:50.130977 (+1043740us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540035a7d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:50.131103 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2146795904}
I20260709 12:13:49.987763 26018 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.135704 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.143860 26035 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.144474 26035 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:49.926484 25962 rpcz_store.cc:276] 0709 12:13:46.753956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.756394 (+  2438us) service_pool.cc:224] Handling call
0709 12:13:47.212710 (+456316us) ts_tablet_manager.cc:934] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
0709 12:13:48.541950 (+1329240us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:49.198460 (+656510us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:49.198575 (+   115us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:49.632980 (+434405us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:49.663426 (+ 30446us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:49.753058 (+ 89632us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:49.762995 (+  9937us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":23934,"tablet-copy.queue_time_us":319111,"thread_start_us":701686,"threads_started":2}
I20260709 12:13:50.145309 26036 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:50.146162 25962 tablet_copy_client.cc:806] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:50.145048 26018 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:50.102697 25906 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:13:50.154670 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.155402 23781 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:50.161720 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.164726 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.124289 25925 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:13:50.163487 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:50.168752 23825 leader_election.cc:336] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.086s (SENT)
I20260709 12:13:50.157074 25861 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:50.153270 26032 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:50.157963 25962 tablet_copy_client.cc:670] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:50.175153 23776 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:50.175572 23776 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.147626 25843 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.100935 26020 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.151105 25936 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:13:50.166958 25906 consensus_queue.cc:1243] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:13:50.193476 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:50.176370 23776 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.174965 25861 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.204411 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:50.162902 26011 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 12:13:50.216957 23825 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.810s (SENT)
W20260709 12:13:50.217701 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 298298us
I20260709 12:13:50.185192 26032 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:50.203826 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 175488us
I20260709 12:13:50.217687 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:50.193030 25979 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.222165 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.222441 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.917s (SENT)
I20260709 12:13:50.222374 23776 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:50.228926 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65505.8us
I20260709 12:13:50.230109 26011 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:50.230693 26011 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.231657 26011 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.234511 23751 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.895s (SENT)
W20260709 12:13:50.235112 23751 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.935s (SENT)
I20260709 12:13:50.235462 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:50.219208 25861 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:50.243564 23825 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.096s (SENT)
I20260709 12:13:50.221050 26032 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.246152 23776 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:50.221282 26050 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:50.248579 25843 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:50.245435 23825 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.169s (SENT)
I20260709 12:13:50.253010 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.254343 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.262279 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:50.263460 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:50.264101 23899 connection.cc:256] RPC call timeout handler was delayed by -1.2440010639998036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
W20260709 12:13:50.264782 23899 connection.cc:256] RPC call timeout handler was delayed by -1.2377010639997934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 12:13:50.265147 23899 connection.cc:256] RPC call timeout handler was delayed by -1.1774010639999233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 12:13:50.254992 23776 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:50.239001 26057 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:50.270562 26018 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:50.288909 23925 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:50.289922 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:50.293256 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.295799 23933 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:50.292351 23777 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:50.251709 25962 tablet_copy_client.cc:538] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:50.293263 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.298537 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.302117 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.303541 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:50.303306 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.297073 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 61985.8us
W20260709 12:13:50.297474 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.311852 23751 leader_election.cc:336] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 2.078s (SENT)
W20260709 12:13:50.311210 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.313141 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.313268 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.316447 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.241578 25979 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.319245 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.330488 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:50.331974 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 64530.2us
I20260709 12:13:50.297986 25925 consensus_queue.cc:1243] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:50.299293 26061 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:50.332106 23859 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:50.333488 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:50.334200 23859 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:50.336299 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.339883 23785 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:50.342727 23860 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:13:50.323956 25967 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.345902 25967 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.317198 26036 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:50.348493 23785 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:50.298143 25796 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:13:50.352749 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.354065 26046 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:50.354712 23825 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.562s (SENT)
I20260709 12:13:50.360153 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:50.358865 23927 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:50.363037 23859 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.336967 26061 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.371042 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:50.374097 23859 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:50.376065 23852 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:50.369127 26056 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.377053 23852 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:50.358108 25971 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:50.381559 23778 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:50.360361 26035 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.378115 25887 tablet_copy_client.cc:538] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:50.395345 23927 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:50.383800 25979 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:50.377888 26050 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:13:50.385094 25971 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:50.416400 23825 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.105s (SENT)
W20260709 12:13:50.430649 23825 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.144s (SENT)
I20260709 12:13:50.431077 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:50.428367 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:50.438400 23859 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:50.440109 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.440801 23778 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:50.441859 23854 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:50.397949 25957 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:50.451392 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.393312 25925 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:13:50.454339 23859 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:50.455349 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.455703 23856 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:50.458002 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:50.427891 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:50.453537 25894 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:13:50.439337 26011 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:13:50.453557 26056 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:50.443209 26018 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:13:50.445967 25967 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.478508 25971 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:50.463301 26069 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 12:13:50.468278 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 200861us
I20260709 12:13:50.469045 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.428056 25973 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.486670 23899 connection.cc:256] RPC call timeout handler was delayed by -1.0361010639999222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 12:13:50.487459 23858 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:50.488006 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.495034 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.455294 26032 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.510737 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 360) took 1885 ms. Trace:
W20260709 12:13:50.511243 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.427583 26061 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.593217 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.592376 23899 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.572s (SENT)
I20260709 12:13:50.537602 23781 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:50.511238 25176 rpcz_store.cc:276] 0709 12:13:48.601847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.602291 (+   444us) service_pool.cc:224] Handling call
0709 12:13:48.634290 (+ 31999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.533400 (+899110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.486569 (+953169us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400703354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:50.486772 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054813056}
W20260709 12:13:50.581698 23825 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.960s (SENT)
W20260709 12:13:50.510334 26001 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.002s
W20260709 12:13:50.539786 26070 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:13:50.508689 26011 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:50.602068 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:13:50.604802 23927 raft_consensus.cc:606] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:50.605180 23927 raft_consensus.cc:614] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:50.562342 25843 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.604156 23851 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:50.562553 26027 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:50.603571 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 206799us
W20260709 12:13:50.602918 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.793s (SENT)
I20260709 12:13:50.608367 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:50.593044 25973 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.471187 26049 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.610601 26018 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:50.612463 26065 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:50.613821 25925 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:50.614538 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.607007 26011 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.615479 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:50.651139 23858 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.601508 26032 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.616164 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:50.618645 23899 leader_election.cc:336] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.870s (SENT)
I20260709 12:13:50.616142 25843 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.622473 26063 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:50.629367 23927 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:50.630061 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:50.658279 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:13:50.658488 23925 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:50.630736 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:50.616714 26027 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.631637 23934 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:50.652186 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:50.661602 23825 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.141s (SENT)
W20260709 12:13:50.663745 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45129.1us
I20260709 12:13:50.616809 26061 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.666484 23852 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:50.666702 23859 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:50.669339 25887 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
W20260709 12:13:50.667181 23899 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.939s (SENT)
I20260709 12:13:50.668933 25861 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.670284 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:50.650810 25973 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.657779 26063 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.665771 25978 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:50.619234 26001 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:13:50.675908 26018 consensus_queue.cc:1243] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:50.670392 25979 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.704268 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2058) took 1128 ms. Trace:
I20260709 12:13:50.675112 26032 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:50.705327 23930 rpcz_store.cc:276] 0709 12:13:49.575735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.624960 (+1049225us) service_pool.cc:224] Handling call
0709 12:13:50.625287 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.703903 (+ 78616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.703934 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.703982 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:50.704017 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.704083 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.704206 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51619,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":24438,"thread_start_us":61917,"threads_started":1}
I20260709 12:13:50.688158 25973 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.705968 26011 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.708180 26078 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:13:50.717059 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.720690 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.722056 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.723176 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.709332 23899 leader_election.cc:336] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.946s (SENT)
I20260709 12:13:50.726202 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:50.710364 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.716305 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.728886 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 100334us
I20260709 12:13:50.732057 23858 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:50.733732 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:50.733770 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.736552 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:50.743112 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.712666 25973 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:50.749053 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.750296 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.752828 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.753827 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:50.753978 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.716277 26011 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:50.760479 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.761159 23779 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:50.761905 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.762300 23779 raft_consensus.cc:2376] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5. Re-sending same reply.
I20260709 12:13:50.762894 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.751355 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.765597 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:50.766101 23783 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:50.714238 26062 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:50.766157 23934 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.708443 26032 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.767100 23925 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:50.768162 23928 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:50.768778 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.783622 25176 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:50.801829 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.736016 25977 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:13:50.782392 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 52928.3us
W20260709 12:13:50.782015 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72718.4us
I20260709 12:13:50.805145 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:50.805760 23899 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.993s (SENT)
I20260709 12:13:50.806161 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:50.803303 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:50.803947 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:50.810364 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:50.769503 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:50.814329 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:50.708618 25979 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.815372 23933 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:13:50.752015 26084 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:50.768800 26062 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.826663 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 119) took 1375 ms. Trace:
I20260709 12:13:50.828102 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:50.823727 23934 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:50.827105 23932 rpcz_store.cc:276] 0709 12:13:49.451200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.825887 (+1374687us) service_pool.cc:224] Handling call
0709 12:13:50.826179 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.826355 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.826379 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.826419 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:50.826457 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.826508 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.826606 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:50.832723 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:50.810752 25925 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:50.833106 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2053) took 1260 ms. Trace:
I20260709 12:13:50.801141 25858 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:13:50.835810 23926 rpcz_store.cc:276] 0709 12:13:49.573122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.818985 (+1245863us) service_pool.cc:224] Handling call
0709 12:13:50.832597 (+ 13612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.832783 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.832809 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.832851 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:50.832890 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.832940 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.833047 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:50.839097 26049 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.839378 25861 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.839301 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:50.839890 25843 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:50.840629 25843 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:50.751996 26086 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 12:13:50.837386 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:50.845393 26087 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:50.846266 26065 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:50.846004 26087 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.846704 26065 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.847224 26018 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:50.847632 26018 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:50.848223 26019 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 1.595s	user 0.005s	sys 0.010s
I20260709 12:13:50.836198 25967 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:50.837519 26080 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.848461 26018 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.856621 23783 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:50.859692 23783 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:50.860986 26082 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:13:50.861378 25906 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:50.861753 25906 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:50.862517 25906 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.863911 26029 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 1.390s	user 0.005s	sys 0.001s
I20260709 12:13:50.864332 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 267) took 3615 ms. Trace:
I20260709 12:13:50.866175 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:50.860899 25962 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:13:50.875537 25906 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.877300 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:50.864699 23680 rpcz_store.cc:276] 0709 12:13:47.249273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.252529 (+  3256us) service_pool.cc:224] Handling call
0709 12:13:50.864258 (+3611729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:48.370740 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:48.371291 (+   551us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:48.371328 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:13:48.371347 (+    19us) tablet.cc:662] Decoding operations
0709 12:13:48.458166 (+ 86819us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:48.482221 (+ 24055us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:48.484900 (+  2679us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:13:48.586942 (+102042us) tablet.cc:701] Row locks acquired
0709 12:13:48.589997 (+  3055us) write_op.cc:260] PREPARE: finished
0709 12:13:48.597507 (+  7510us) write_op.cc:270] Start()
0709 12:13:48.612754 (+ 15247us) write_op.cc:276] Timestamp: P: 1783599228595434 usec, L: 0
0709 12:13:48.614928 (+  2174us) op_driver.cc:348] REPLICATION: starting
0709 12:13:48.763840 (+148912us) log.cc:844] Serialized 1360 byte log entry
0709 12:13:49.655543 (+891703us) op_driver.cc:464] REPLICATION: finished
0709 12:13:50.085002 (+429459us) write_op.cc:301] APPLY: starting
0709 12:13:50.103552 (+ 18550us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:50.126348 (+ 22796us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:50.156929 (+ 30581us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:50.226342 (+ 69413us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:50.226558 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:13:50.226606 (+    48us) write_op.cc:312] APPLY: finished
0709 12:13:50.226858 (+   252us) log.cc:844] Serialized 35 byte log entry
0709 12:13:50.309381 (+ 82523us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:50.309491 (+   110us) write_op.cc:454] Released schema lock
0709 12:13:50.309532 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:50.331841 (+ 22309us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:50.335576 (+  3735us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1173,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":414737,"apply.run_cpu_time_us":31194,"apply.run_wall_time_us":243168,"num_ops":3,"prepare.queue_time_us":150531,"prepare.run_cpu_time_us":4616,"prepare.run_wall_time_us":414907,"replication_time_us":1012097,"thread_start_us":451494,"threads_started":3,"wal-append.queue_time_us":82572}]]}
W20260709 12:13:50.881595 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.883388 26050 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:50.855712 25967 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:50.884215 26050 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:50.885255 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.874796 26080 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:50.874636 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:50.872205 23783 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:50.888335 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.895627 26027 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:50.899117 25171 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:13:50.906865 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.851187 26039 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
W20260709 12:13:50.925047 26081 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.096s	user 0.001s	sys 0.003s
I20260709 12:13:50.768798 26063 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.838016 26032 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.866854 26062 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:50.929870 26062 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.884608 25967 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:50.914716 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.692s (SENT)
I20260709 12:13:50.917238 23783 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:50.900578 25979 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:50.866300 26049 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:13:50.915907 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:50.946813 26018 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:50.973338 23825 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.117s (SENT)
I20260709 12:13:50.847669 26065 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:50.963042 26087 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:51.002236 25864 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:50.977439 23783 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:13:51.022011 26076 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 12:13:51.032163 23776 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:51.011266 25864 tablet_bootstrap.cc:492] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:51.036459 23776 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:51.040144 26065 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:51.042727 25168 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:13:51.012588 26032 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:51.057050 25864 ts_tablet_manager.cc:1403] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 2.288s	user 0.040s	sys 0.046s
W20260709 12:13:51.059737 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.026268 26061 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.067811 26061 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.073958 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:51.030952 26049 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:51.052016 26079 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.262s	user 0.000s	sys 0.004s
I20260709 12:13:51.024319 26080 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:51.097262 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.102849 26052 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:51.103654 26052 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:51.104230 26052 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:51.114547 26044 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:13:51.106348 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:51.097857 26049 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:51.185468 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.175324 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2066) took 1181 ms. Trace:
I20260709 12:13:51.192605 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:51.193030 25967 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.187497 23778 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:51.198199 23777 rpcz_store.cc:276] 0709 12:13:49.993781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.104761 (+110980us) service_pool.cc:224] Handling call
0709 12:13:51.175139 (+1070378us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003a4654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:51.175256 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340614528}
I20260709 12:13:51.199424 25971 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:13:51.224747 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.225591 25971 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:51.230401 25971 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.222779 26061 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:51.271006 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.251554 25967 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:51.266327 25971 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:51.292279 23749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.014s	sys 0.015s
W20260709 12:13:51.294659 23749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.015s	sys 0.015s
W20260709 12:13:51.294967 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4945572320000338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 12:13:51.295249 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4684572319999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 12:13:51.295516 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4108572319998984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 12:13:51.299788 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4081572319998941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 12:13:51.301064 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4072572319998926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 12:13:51.302381 23749 connection.cc:256] RPC call timeout handler was delayed by -3.3811572320000778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 12:13:51.304767 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.305751 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 499983us
W20260709 12:13:51.306607 23899 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.013s (SENT)
I20260709 12:13:51.302229 26052 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:51.303862 23749 connection.cc:256] RPC call timeout handler was delayed by -3.3469572320000225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 12:13:51.308619 23749 connection.cc:256] RPC call timeout handler was delayed by -3.2956572319999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
I20260709 12:13:51.309358 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:51.310910 23929 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.011s	sys 0.022s
W20260709 12:13:51.309654 23749 connection.cc:256] RPC call timeout handler was delayed by -3.2893572319999294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1769s for a response
W20260709 12:13:51.311962 23749 connection.cc:256] RPC call timeout handler was delayed by -3.21105723200003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 12:13:51.311380 23929 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.011s	sys 0.022s
W20260709 12:13:51.314049 23749 connection.cc:256] RPC call timeout handler was delayed by -3.1759572319999734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 12:13:51.314558 23749 connection.cc:256] RPC call timeout handler was delayed by -3.143557231999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 12:13:51.314918 23749 connection.cc:256] RPC call timeout handler was delayed by -3.0112572319999344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 12:13:51.315272 23749 connection.cc:256] RPC call timeout handler was delayed by -2.8753572319999421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 12:13:51.315989 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.322239 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2078) took 1136 ms. Trace:
I20260709 12:13:51.322706 23781 rpcz_store.cc:276] 0709 12:13:50.186273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.199161 (+ 12888us) service_pool.cc:224] Handling call
0709 12:13:51.322009 (+1122848us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400093154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:51.322164 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712356736}
I20260709 12:13:51.270040 25864 raft_consensus.cc:359] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:51.330152 26072 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.855s	user 0.014s	sys 0.000s
I20260709 12:13:51.332484 26050 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.323840 26061 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:51.339352 26050 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:51.337628 26063 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:51.359740 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.373987 23776 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:51.326671 25864 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:51.384243 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.385098 26061 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:51.387015 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 80443.2us
W20260709 12:13:51.386731 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.000s	sys 0.011s
W20260709 12:13:51.390468 23899 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.015s (SENT)
W20260709 12:13:51.390887 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.000s	sys 0.011s
W20260709 12:13:51.390528 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.003s	sys 0.049s
W20260709 12:13:51.391373 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 637420us
I20260709 12:13:51.379830 26065 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:51.393543 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.003s	sys 0.049s
I20260709 12:13:51.379133 26056 raft_consensus.cc:476] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 12:13:51.393635 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:51.394834 23934 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:51.398029 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.394351 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:51.386323 26080 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:51.399153 23933 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:51.386562 26049 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:51.402012 23934 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:13:51.402482 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.427327 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:51.431790 26032 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:13:51.440459 23933 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:51.442106 26093 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:51.443473 26061 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:51.445117 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:51.445662 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.006s	sys 0.007s
W20260709 12:13:51.455034 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.006s	sys 0.007s
W20260709 12:13:51.455471 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 647996us
I20260709 12:13:51.427305 26065 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 12:13:51.459947 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.478255 23934 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:51.471174 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.454599 26061 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:51.513919 26044 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:51.515262 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 44184.6us
I20260709 12:13:51.516884 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.464704 26032 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:51.523478 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.531208 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:51.532083 23856 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:51.532436 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.533120 23776 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:51.493675 26065 consensus_queue.cc:1243] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:13:51.537781 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.537441 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 448774us
W20260709 12:13:51.519048 25836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.013s	sys 0.010s
W20260709 12:13:51.500510 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 404399us
I20260709 12:13:51.403144 25864 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:51.591430 23860 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:51.592159 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68765.3us
I20260709 12:13:51.508126 25973 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:51.596352 23779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.037s	sys 0.080s
I20260709 12:13:51.600401 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:51.601898 23779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.037s	sys 0.081s
I20260709 12:13:51.536007 26044 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:51.590260 25836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.013s	sys 0.011s
W20260709 12:13:51.537396 26086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.008s	sys 0.009s
I20260709 12:13:51.590227 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:51.604053 26080 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.639645 26086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.008s	sys 0.010s
I20260709 12:13:51.636462 26098 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:51.606854 25973 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:51.607656 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.535845 26052 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:51.593454 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56155.1us
I20260709 12:13:51.606292 23776 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
W20260709 12:13:51.535544 25925 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.002s	sys 0.016s
I20260709 12:13:51.634044 26061 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:51.646060 26050 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.676146 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68579.9us
I20260709 12:13:51.668159 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:51.685065 23851 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:51.680940 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:51.682935 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:51.686412 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:51.678201 26098 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:51.689400 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.007s	sys 0.014s
I20260709 12:13:51.647578 25171 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:13:51.689690 23782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.009s	sys 0.013s
I20260709 12:13:51.685431 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:51.705212 23782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.014s
I20260709 12:13:51.687942 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:51.701941 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.007s	sys 0.014s
W20260709 12:13:51.690073 26025 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.012s	sys 0.008s
I20260709 12:13:51.663430 25967 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:13:51.678277 26083 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:51.689579 25906 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.813s	user 0.010s	sys 0.003s
I20260709 12:13:51.653954 26080 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:51.696713 25925 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.003s	sys 0.016s
W20260709 12:13:51.704478 26011 consensus_meta.cc:287] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.922s	user 0.013s	sys 0.000s
I20260709 12:13:51.714704 26099 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:51.678925 25979 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 12:13:51.709151 26025 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.013s	sys 0.008s
I20260709 12:13:51.800926 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.715915 26057 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:51.713507 26061 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:51.797731 23860 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:51.713905 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:51.722935 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.767630 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 575) took 1146 ms. Trace:
I20260709 12:13:51.786155 26080 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:51.696475 26050 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.815964 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 103) took 1495 ms. Trace:
I20260709 12:13:51.804296 23783 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:51.699451 26056 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:13:51.716420 26097 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.716924 25925 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:51.763609 26052 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:51.714872 25977 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:51.809113 25894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.017s	sys 0.045s
I20260709 12:13:51.826171 23779 rpcz_store.cc:276] 0709 12:13:50.320998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.810020 (+489022us) service_pool.cc:224] Handling call
0709 12:13:50.810451 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.817702 (+  7251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.817731 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.834027 (+ 16296us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.603327 (+769300us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:51.603380 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.603449 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.815740 (+212291us) raft_consensus.cc:1685] finished
0709 12:13:51.815760 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.815889 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149083,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":32213,"thread_start_us":747283,"threads_started":1,"wal-append.queue_time_us":963348}
I20260709 12:13:51.842120 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.842762 23779 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:51.823320 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:51.828302 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 230277us
I20260709 12:13:51.823984 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.825253 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 106) took 1067 ms. Trace:
I20260709 12:13:51.847265 23776 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:51.830744 23851 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:51.837945 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.824607 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.825012 23857 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:51.848340 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:51.813576 25173 rpcz_store.cc:276] 0709 12:13:50.623271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.651734 (+ 28463us) service_pool.cc:224] Handling call
0709 12:13:51.639156 (+987422us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:51.682836 (+ 43680us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68df8c 00007fb2aa68def6 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:51.763664 (+ 80828us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10619008}
I20260709 12:13:51.812803 26101 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:51.851766 26090 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:51.852172 26090 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:51.852509 26018 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.860s	user 0.008s	sys 0.004s
I20260709 12:13:51.845108 23782 rpcz_store.cc:276] 0709 12:13:50.757630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.110737 (+353107us) service_pool.cc:224] Handling call
0709 12:13:51.111087 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.111447 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.111481 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.114219 (+  2738us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.705598 (+591379us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:51.705636 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.705699 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.825053 (+119354us) raft_consensus.cc:1685] finished
0709 12:13:51.825071 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.825196 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39,"raft.queue_time_us":105,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":26330,"thread_start_us":590910,"threads_started":1,"wal-append.queue_time_us":682771}
W20260709 12:13:51.823530 25971 consensus_meta.cc:287] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.520s	user 0.005s	sys 0.001s
I20260709 12:13:51.854409 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.816044 26050 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:51.861406 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.856143 23782 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:51.866514 23860 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:13:51.868177 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2100) took 1326 ms. Trace:
I20260709 12:13:51.870040 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.870851 23782 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:51.872489 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:51.873574 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 226542us
I20260709 12:13:51.868907 23783 rpcz_store.cc:276] 0709 12:13:50.541278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.017470 (+476192us) service_pool.cc:224] Handling call
0709 12:13:51.017799 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.826296 (+808497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.867937 (+ 41641us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400092c54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:51.868108 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633968384}
I20260709 12:13:51.875362 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.852536 26090 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:51.863230 25894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.017s	sys 0.045s
I20260709 12:13:51.813896 26057 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.877167 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:51.822063 26099 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 12:13:51.885676 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.878827 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 211799us
I20260709 12:13:51.884418 23785 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:51.886650 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.887049 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.821012 26063 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:13:51.885421 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:51.888005 23858 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:51.888363 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.889042 23783 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:13:51.901156 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.850088 26052 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.908680 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.909160 25979 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:51.915279 23778 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:51.917824 23777 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:51.919330 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:51.920212 26050 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:51.920750 26050 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:51.921717 26050 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:51.922197 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:51.923029 26020 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:51.923419 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:51.923826 26020 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:51.925272 25971 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:51.926285 25971 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:51.926764 25971 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:51.926889 25973 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:51.878559 25836 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:51.928898 23749 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:51.934468 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:51.930125 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.931736 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.936551 23784 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:51.936885 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:51.937469 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.939105 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.939888 23783 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:51.939821 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:51.941610 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.843016 26097 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:51.944177 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.943940 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.951166 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.952198 23783 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:51.953428 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.954686 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.958338 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.918287 26099 consensus_queue.cc:1243] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:13:51.928076 25864 ts_tablet_manager.cc:1434] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.835s	user 0.013s	sys 0.005s
I20260709 12:13:51.962574 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:51.966598 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:51.925024 26020 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:51.970446 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.982038 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.977550 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:51.982414 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.983449 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.983711 23824 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:51.983824 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.931257 26095 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:51.943401 26086 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:51.942850 26018 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.002678 23926 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:52.004135 23931 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:51.957335 26011 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:52.007840 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.008210 23933 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:51.943727 26101 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:51.997323 23925 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:51.998296 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2069) took 1232 ms. Trace:
W20260709 12:13:52.015467 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 79648.5us
I20260709 12:13:51.999011 23932 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:52.000046 23928 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:13:51.987251 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:52.013908 23929 rpcz_store.cc:276] 0709 12:13:50.765739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.766985 (+  1246us) service_pool.cc:224] Handling call
0709 12:13:50.806208 (+ 39223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.313132 (+506924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.313159 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.320101 (+  6942us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.713954 (+393853us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:51.713989 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.714063 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.998059 (+283996us) raft_consensus.cc:1685] finished
0709 12:13:51.998076 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.998213 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33900,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":843227,"threads_started":2,"wal-append.queue_time_us":522485}
I20260709 12:13:52.017894 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.019268 23778 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:52.021310 23858 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:51.981887 25979 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:13:52.023072 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:52.024001 26097 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.025024 25979 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.025754 23929 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.025537 25979 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:51.992074 25864 ts_tablet_manager.cc:1467] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Tablet startup took 911ms
I20260709 12:13:52.028642 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.029327 23927 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:52.030529 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:51.959738 26080 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 12:13:52.039994 26097 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:52.041858 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.041277 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:51.928722 25925 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:51.958105 25836 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:51.944713 26098 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:13:52.018023 26018 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.052047 23781 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:52.053223 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.053786 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.054402 23781 raft_consensus.cc:2376] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
W20260709 12:13:52.042428 26062 consensus_meta.cc:287] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.104s	user 0.013s	sys 0.002s
I20260709 12:13:51.955664 25971 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.062098 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.062743 23781 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4. Re-sending same reply.
I20260709 12:13:52.063709 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.064260 23781 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:52.065166 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.066331 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.066924 23781 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:52.050143 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:52.068924 26052 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.045562 26097 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.069759 26097 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:52.058298 26018 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.019830 26121 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:52.071946 26018 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.074182 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:52.075114 23783 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:52.077355 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:52.079742 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.090109 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:52.071831 26115 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:13:52.090884 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.015808 26095 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.094754 23932 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Committing config change with OpId 5.5: config changed from index -1 to 5, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:13:52.117448 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.118187 23925 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:52.126722 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.114607 25979 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.129614 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.142194 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.142553 23928 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:52.132455 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.153992 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.158375 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:52.159406 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:52.160126 23926 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:13:52.161211 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.161746 23934 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.129592 26098 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:52.164908 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.165629 23859 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:52.166307 26098 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:52.163415 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 107) took 1404 ms. Trace:
I20260709 12:13:52.125653 26057 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.168111 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.168757 23781 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:52.164446 23926 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:52.138015 26120 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:52.132320 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:52.170820 23781 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:52.174762 23781 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:52.125016 26062 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:52.167688 23780 rpcz_store.cc:276] 0709 12:13:50.759264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.115062 (+355798us) service_pool.cc:224] Handling call
0709 12:13:51.143415 (+ 28353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.143796 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.143822 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.176539 (+ 32717us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.424044 (+247505us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800293bc0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:13:51.838152 (+414108us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:51.838189 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.838253 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.158274 (+320021us) raft_consensus.cc:1685] finished
0709 12:13:52.158293 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.163336 (+  5043us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13711,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4856,"spinlock_wait_cycles":4770688,"thread_start_us":413927,"threads_started":1,"wal-append.queue_time_us":730882}
I20260709 12:13:52.177747 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.144304 26095 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.174134 26018 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:13:52.188058 26099 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.186379 23781 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:52.168560 26065 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:52.229602 23781 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:52.232142 26090 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.198346 26062 consensus_queue.cc:1243] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:13:52.189118 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:52.200781 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.234337 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 45312.5us
I20260709 12:13:52.235042 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.236585 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.238184 26083 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:52.239143 26083 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:52.240196 26083 consensus_queue.cc:1243] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:52.240849 26083 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.241673 26083 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.242611 26083 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:52.243515 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:52.245112 26086 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:52.245590 26086 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:52.245045 26095 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:52.248086 26095 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.248607 26095 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.253015 23782 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:13:52.238305 23781 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:13:52.256121 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2096) took 1716 ms (client timeout 1614 ms). Trace:
W20260709 12:13:52.256583 23785 rpcz_store.cc:269] 0709 12:13:50.540113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.716801 (+176688us) service_pool.cc:224] Handling call
0709 12:13:52.255957 (+1539156us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540035acd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:52.256055 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2514961664}
I20260709 12:13:52.257490 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:52.258150 26087 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:52.258556 26087 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:52.033044 25864 ts_tablet_manager.cc:1469] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Trace:
0709 12:13:42.651960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.652127 (+   167us) service_pool.cc:224] Handling call
0709 12:13:42.893070 (+240943us) ts_tablet_manager.cc:934] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
0709 12:13:43.926201 (+1033131us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4c00116940. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aba7ea20 00007fb2aba8fa29 00007fb2aba9013f 00007fb2aba542ab 00007fb2aba81f3c 00007fb2aba82461 00007fb2aba82768 00007fb2b18082eb 00007fb2b272dd3b 00007fb2b2728008 00007fb2b2727405
0709 12:13:45.886021 (+1959820us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:46.767866 (+881845us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:46.817674 (+ 49808us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:47.442998 (+625324us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:47.444534 (+  1536us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:47.444873 (+   339us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:47.445084 (+   211us) inbound_call.cc:177] Queueing success response
0709 12:13:48.768452 (+1323368us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:51.092953 (+2324501us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:51.226186 (+133233us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:51.227203 (+  1017us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":60307,"mutex_wait_us":32698,"raft.queue_time_us":41324,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":16534,"spinlock_wait_cycles":5138688,"tablet-copy.queue_time_us":188372,"tablet-download-pool.queue_time_us":212,"tablet-download-pool.run_cpu_time_us":1427,"tablet-download-pool.run_wall_time_us":418396,"thread_start_us":759926,"threads_started":2}
I20260709 12:13:52.260485 25171 raft_consensus.cc:1217] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:13:52.261456 26062 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:52.262017 26062 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.262564 26062 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.262874 23778 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:52.263275 23778 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:52.265776 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:52.178064 26120 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.185174 26098 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.219231 25836 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.232712 25168 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:52.259344 26087 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.279407 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:52.291213 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.280481 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.267534 23778 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.260937 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:52.300644 23778 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.273110 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:52.275745 23785 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:52.276221 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.275543 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:52.302217 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.302613 23781 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:52.208806 23852 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:52.307171 26083 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.314844 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.316511 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:52.317795 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.290938 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:52.322103 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49088.2us
I20260709 12:13:52.324015 23860 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.307921 26083 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.246776 26086 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.279299 26057 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:52.333613 23931 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:52.291267 26087 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.337433 23855 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:52.291682 26124 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.341156 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.330322 25971 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.322436 26049 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.327924 26050 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:52.296849 26039 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:13:52.331187 25987 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:52.331580 26052 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:52.333216 26098 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.335848 26057 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:52.378651 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.386406 26065 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.387017 26065 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:52.394898 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.351729 26050 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.341900 26099 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:13:52.389153 26065 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.389840 26057 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:52.394500 26101 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 12:13:52.433176 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.419221 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73729.5us
I20260709 12:13:52.400908 26050 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:52.480607 26137 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.131s	user 0.002s	sys 0.001s
W20260709 12:13:52.483119 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.492728 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:52.487803 25987 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:52.488313 26093 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:13:52.375048 26098 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:52.502669 23899 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.109s	user 0.009s	sys 0.066s
I20260709 12:13:52.350565 26124 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:52.503082 23899 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.009s	sys 0.066s
W20260709 12:13:52.505913 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 1.11257e+06us
W20260709 12:13:52.506539 23899 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.045s (SENT)
I20260709 12:13:52.508064 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:52.514822 23859 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:52.539608 25987 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.559150 26124 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.569621 26120 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.514602 26097 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:52.594322 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 126930us
I20260709 12:13:52.568619 26090 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:52.589637 26136 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:13:52.624008 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:52.626793 23781 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
W20260709 12:13:52.635429 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 317674us
I20260709 12:13:52.614810 26093 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.640911 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.618985 26011 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.621726 26083 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:52.642475 26141 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:13:52.657996 25973 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.626612 26095 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:52.637346 26140 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:13:52.669296 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:52.674906 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.668462 26011 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.679412 26011 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.680193 25973 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:52.680817 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.681203 26018 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:52.680879 26093 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.689924 26018 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:52.690955 26097 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:52.691359 26018 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:52.694500 25973 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.694842 26018 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.697192 26098 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.718968 23780 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
W20260709 12:13:52.719573 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.722112 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.726576 26086 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:52.730360 25174 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:52.730685 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:52.731074 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 426253us
I20260709 12:13:52.735136 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:52.735762 23860 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.737507 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:52.744163 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:52.722124 23680 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:52.744875 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 252232us
I20260709 12:13:52.749859 23680 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.751808 23680 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.764173 23680 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.715560 26095 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:52.773509 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 266989us
I20260709 12:13:52.772728 23680 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
W20260709 12:13:52.785131 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 89312.3us
I20260709 12:13:52.784099 23680 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:13:52.795423 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.795765 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.795910 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:52.796466 23933 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:52.796983 23929 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.794845 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.799830 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:52.742553 26117 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:13:52.802258 26117 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:52.773425 26091 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:52.802786 26117 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:52.803062 26091 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.804507 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.773664 26119 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.806887 23929 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:52.803613 26091 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:52.810216 25843 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.815338 23928 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:52.815385 23929 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.794122 23680 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.819216 23680 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.824633 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:52.818336 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:52.804328 26046 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.830040 23860 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:52.830137 26094 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 12:13:52.814177 26111 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.904s	user 0.004s	sys 0.000s
I20260709 12:13:52.828487 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.793375 26068 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:13:52.834020 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:52.841704 23931 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:13:52.834183 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:52.817018 26124 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:13:52.835564 26094 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:52.825719 25843 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.806515 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:52.830070 26046 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:52.870462 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.836900 23680 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 3, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 3 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.837669 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:52.840618 23928 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.793725 25973 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:13:52.798514 26097 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:13:52.832655 26107 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:13:52.844286 26049 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:52.906095 26062 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.610s	user 0.008s	sys 0.004s
I20260709 12:13:52.826686 26093 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:52.906662 26062 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:52.846804 26094 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:52.877918 23680 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:52.810539 26117 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:52.890848 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.007s	sys 0.027s
I20260709 12:13:52.885850 26107 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:52.887220 26098 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:52.965086 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.158783 23927 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:52.932811 23785 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:52.990265 25113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.015s	sys 0.021s
W20260709 12:13:53.160001 25113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.016s	sys 0.032s
W20260709 12:13:52.975147 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.158483 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:53.158412 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 324456us
I20260709 12:13:53.161548 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:53.162546 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.014s	sys 0.029s
W20260709 12:13:53.162972 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 843682us
W20260709 12:13:53.068009 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.967494 25979 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.032s	sys 0.036s
W20260709 12:13:53.164728 25979 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.032s	sys 0.050s
W20260709 12:13:53.077700 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.166599 23934 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:53.174945 26098 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:52.999773 23926 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.012s	sys 0.032s
W20260709 12:13:53.177111 23926 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.012s	sys 0.038s
I20260709 12:13:53.166579 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2088) took 1093 ms. Trace:
I20260709 12:13:53.178723 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:53.179385 23933 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:53.179153 23932 rpcz_store.cc:276] 0709 12:13:52.073125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.073281 (+   156us) service_pool.cc:224] Handling call
0709 12:13:52.094219 (+ 20938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.094628 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:52.125622 (+ 30994us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.479921 (+354299us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:52.742629 (+262708us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:52.744010 (+  1381us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.744161 (+   151us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:52.744193 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.744253 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.887665 (+143412us) raft_consensus.cc:1685] finished
0709 12:13:52.912249 (+ 24584us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.166426 (+254177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424922,"raft.queue_time_us":421273,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":3652,"spinlock_wait_cycles":29854976,"thread_start_us":190726,"threads_started":1,"wal-append.queue_time_us":216176}
I20260709 12:13:53.182552 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:52.962481 26151 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:52.912670 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.173187 26080 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:53.160147 26121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.001s	sys 0.017s
I20260709 12:13:53.212217 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2063) took 2432 ms. Trace:
I20260709 12:13:53.162249 23783 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
W20260709 12:13:53.215298 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.172049 26094 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.185194 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:53.173604 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 335985us
I20260709 12:13:52.956068 26046 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:53.217285 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:53.225567 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.181494 25887 log.cc:927] Time spent T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:13:53.159693 26011 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.213621 23930 rpcz_store.cc:276] 0709 12:13:50.763985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.764101 (+   116us) service_pool.cc:224] Handling call
0709 12:13:50.807460 (+ 43359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.310522 (+503062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.310554 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.313940 (+  3386us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.707272 (+393332us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:51.707312 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.707374 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.810774 (+1103400us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:52.964190 (+153416us) raft_consensus.cc:1685] finished
0709 12:13:52.964216 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.195795 (+231579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9461,"raft.queue_time_us":48788,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8927,"spinlock_wait_cycles":43327744,"thread_start_us":866472,"threads_started":2,"wal-append.queue_time_us":534680}
I20260709 12:13:52.957569 25843 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.181072 26107 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:53.233740 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:53.181172 26065 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.136294 26152 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:53.210855 25962 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:13:53.155388 26124 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:52.886507 26050 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.197008 26049 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.182199 25925 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:53.223501 26057 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.005s
I20260709 12:13:53.266937 26057 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.269178 26057 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.242442 23856 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
W20260709 12:13:53.238843 26115 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:13:53.229935 26093 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:53.272297 26093 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.272782 26093 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.230505 26151 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:53.196112 25973 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:13:53.231405 26080 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:53.251250 26152 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:53.273613 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.274385 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:53.241951 26117 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.276683 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:53.235667 26121 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.287412 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:53.290831 23932 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:13:53.301748 26152 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.306942 26158 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:53.307548 26158 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.307992 26158 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:53.310711 26018 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
I20260709 12:13:53.310506 26062 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.312332 25887 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:13:53.313591 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.314181 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 788) took 1274 ms. Trace:
I20260709 12:13:53.284824 26151 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.242177 26011 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.254333 26124 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.324100 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:53.252651 26065 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.324558 23927 rpcz_store.cc:276] 0709 12:13:52.039873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.040008 (+   135us) service_pool.cc:224] Handling call
0709 12:13:53.158607 (+1118599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.159196 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.313952 (+154756us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002a4250. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:53.314109 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571760384}
I20260709 12:13:53.342589 23932 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:13:53.295926 25977 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:53.277227 25925 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:53.265381 26035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.007s	sys 0.034s
W20260709 12:13:53.376770 25112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.032s
W20260709 12:13:53.377084 23777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.011s	sys 0.016s
W20260709 12:13:53.377408 23777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.011s	sys 0.016s
I20260709 12:13:53.276517 26049 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:53.347381 26086 consensus_meta.cc:287] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.604s	user 0.008s	sys 0.004s
W20260709 12:13:53.360918 26146 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.006s	sys 0.034s
I20260709 12:13:53.372161 25977 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:53.389638 25112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.001s	sys 0.032s
I20260709 12:13:53.392685 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:53.359210 26098 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:53.378387 26035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.007s	sys 0.035s
W20260709 12:13:53.386540 25978 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.029s
I20260709 12:13:53.288267 26107 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.297237 26121 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.407886 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.400295 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2091) took 1258 ms. Trace:
W20260709 12:13:53.391811 26146 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.006s	sys 0.034s
I20260709 12:13:53.397302 25977 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:53.420526 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 238442us
W20260709 12:13:53.405200 25978 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.007s	sys 0.029s
W20260709 12:13:53.424917 25113 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 2.214s (SENT)
I20260709 12:13:53.440697 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:53.444456 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.387748 26080 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:53.419493 26106 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.013s	sys 0.069s
I20260709 12:13:53.409994 25887 tablet_bootstrap.cc:492] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:53.425431 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:53.464711 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:53.440879 26091 consensus_meta.cc:287] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.003s
W20260709 12:13:53.373608 25987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.011s	sys 0.026s
I20260709 12:13:53.402596 26025 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:53.428314 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:53.421084 26098 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.433665 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.436406 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:53.437104 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.437759 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.421296 26086 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.450768 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.458122 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:53.461537 23899 connection.cc:256] RPC call timeout handler was delayed by -2.7872098369998639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
W20260709 12:13:53.465560 26106 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.013s	sys 0.070s
I20260709 12:13:53.476984 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 2347) took 1215 ms. Trace:
I20260709 12:13:53.447057 26035 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:53.449383 26157 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:53.479316 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 296485us
I20260709 12:13:53.448668 26121 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.529418 25925 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.529119 26157 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.534858 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:53.490574 23859 rpcz_store.cc:276] 0709 12:13:52.268921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.269044 (+   123us) service_pool.cc:224] Handling call
0709 12:13:53.470606 (+1201562us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000cc6d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:53.470708 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450322688}
W20260709 12:13:53.477886 25987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.011s	sys 0.026s
I20260709 12:13:53.541385 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.489916 25887 ts_tablet_manager.cc:1403] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 2.821s	user 0.034s	sys 0.026s
I20260709 12:13:53.531729 26157 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.465528 26050 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.487980 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:53.468152 26018 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.549984 23779 raft_consensus.cc:2376] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:13:53.557256 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.514907 26094 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.557940 23779 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:13:53.490593 23899 connection.cc:256] RPC call timeout handler was delayed by -2.775509836999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:13:53.558892 23899 connection.cc:256] RPC call timeout handler was delayed by -2.6873098369999298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
W20260709 12:13:53.559410 23899 leader_election.cc:336] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.890s (SENT)
W20260709 12:13:53.479210 26052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.019s	sys 0.079s
W20260709 12:13:53.560214 26052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.020s	sys 0.079s
I20260709 12:13:53.560530 26052 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:53.561528 26052 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.492489 25973 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.563489 25973 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.565884 25973 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.492900 26011 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.492930 26154 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.567886 26154 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.493712 26152 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:53.494179 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.494554 26068 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:53.496847 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:53.570051 26152 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.497921 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:53.571241 23858 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:53.498847 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.570209 23784 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:53.499523 26065 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.499701 23857 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:13:53.500317 26153 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:13:53.516093 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.572829 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:53.517817 26101 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.573421 23784 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4. Re-sending same reply.
I20260709 12:13:53.570233 26068 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.573782 26101 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.523914 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.473508 26091 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.525810 26124 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.528309 26049 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.542133 23783 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:53.484956 26025 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.551728 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:53.485670 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 268449us
I20260709 12:13:53.490118 26098 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.532729 26086 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:53.713194 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.666038 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 136743us
I20260709 12:13:53.620759 23860 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:53.559818 26018 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.749682 26018 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.593261 25168 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:53.720041 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 185250us
W20260709 12:13:53.751147 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.630131 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:53.645358 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 167181us
I20260709 12:13:53.753010 23777 raft_consensus.cc:2376] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:13:53.740063 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:53.719748 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 160379us
W20260709 12:13:53.692193 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 187881us
W20260709 12:13:53.753980 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.894s (SENT)
I20260709 12:13:53.573200 23778 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:53.618393 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:53.764022 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.714679 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:53.617009 26065 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:53.645740 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.712484 26168 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:13:53.710456 26117 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:53.768810 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.095s (SENT)
W20260709 12:13:53.769563 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 278244us
I20260709 12:13:53.773887 23929 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7.
I20260709 12:13:53.737821 26087 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:53.714304 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.736951 26101 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:53.780877 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.156s (SENT)
I20260709 12:13:53.781351 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:53.713732 26094 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:53.768813 26036 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.278s	user 0.009s	sys 0.009s
I20260709 12:13:53.737304 26011 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:53.575445 26068 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.755942 25925 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:53.785502 25887 raft_consensus.cc:359] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:53.786633 26099 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:53.788548 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:53.827553 23777 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:53.788722 26049 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:53.771865 26174 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:13:53.790745 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.793617 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:53.794631 26056 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:53.409489 23926 rpcz_store.cc:276] 0709 12:13:52.141440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.175952 (+ 34512us) service_pool.cc:224] Handling call
0709 12:13:52.176288 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.176509 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:53.177573 (+1001064us) log.cc:844] Serialized 14 byte log entry
0709 12:13:53.399937 (+222364us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:53.399994 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:53.400036 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.400096 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.400218 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114609,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":344,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4396,"thread_start_us":1128637,"threads_started":2}
W20260709 12:13:53.799531 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.773779 26065 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:53.884449 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.798125 25962 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:53.773413 26018 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:53.818665 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:53.771917 26062 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:53.793449 26025 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.844928 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:13:53.808493 26036 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.009s	sys 0.009s
I20260709 12:13:53.702092 26158 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:53.878111 23781 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:53.802809 26117 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:53.881567 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.848918 25887 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:53.823184 26182 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:53.933158 26184 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:53.816944 26154 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:53.930564 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 157038us
W20260709 12:13:53.885954 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 92419.9us
I20260709 12:13:53.874857 26188 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.875799 25973 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.938107 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:53.883816 26152 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:53.882683 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:53.823155 26011 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.845801 25925 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.877420 26099 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.885823 26167 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:13:53.883947 26101 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:53.943544 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 124984us
I20260709 12:13:53.936734 25171 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:53.945955 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 141062us
I20260709 12:13:53.983549 23855 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:53.971117 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:53.996270 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:53.997764 23852 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:53.997668 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:53.992645 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 147808us
I20260709 12:13:53.998832 23854 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:54.003204 26065 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.003988 26065 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:54.013409 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:53.963372 26157 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:13:53.959568 26124 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:53.957752 26094 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:53.938068 26117 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:53.960496 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:54.019430 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:54.026258 23932 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:53.937322 26018 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.004781 26065 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:53.946193 25973 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.023999 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:54.033881 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:54.044004 23928 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:54.034148 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:53.936061 26062 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.012212 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-4e60a1c6520545729d27474d8f237ce5 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:54.004794 26194 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.020125 26080 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.064438 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.000111 26101 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.065084 23777 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:54.054121 25962 tablet_bootstrap.cc:492] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:13:54.071184 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2136) took 1775 ms. Trace:
I20260709 12:13:54.071964 23784 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:13:54.038982 26094 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:54.073743 23784 pending_rounds.cc:85] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 1
I20260709 12:13:54.073403 23780 rpcz_store.cc:276] 0709 12:13:52.295904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.335508 (+ 39604us) service_pool.cc:224] Handling call
0709 12:13:54.070932 (+1735424us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b5400168d54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:54.071125 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865969536}
I20260709 12:13:54.076021 23780 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:13:54.076333 23780 pending_rounds.cc:85] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 2
I20260709 12:13:54.057233 26157 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:53.986649 25887 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:54.082603 23780 pending_rounds.cc:107] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:13:54.087837 23780 raft_consensus.cc:2889] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:54.046824 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:54.087275 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:54.074018 23784 pending_rounds.cc:107] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:13:54.090104 23784 raft_consensus.cc:2889] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:54.079571 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.061838 26080 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:54.067764 26098 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.076998 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-4e60a1c6520545729d27474d8f237ce5 on tablet 4e60a1c6520545729d27474d8f237ce5 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:54.065136 26091 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:53.947489 26188 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.078347 26045 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:13:54.113211 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:13:54.115656 26178 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:13:54.115023 26094 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.087250 26157 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:54.078720 26192 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.104724 25962 ts_tablet_manager.cc:1403] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 3.246s	user 0.061s	sys 0.040s
I20260709 12:13:54.088339 26086 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.097813 26080 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:54.146517 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.127215 23777 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:13:54.106122 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:54.131009 25977 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
I20260709 12:13:54.114257 26098 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:54.153098 26190 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
I20260709 12:13:54.160564 23933 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:13:54.161226 26151 consensus_meta.cc:287] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.788s	user 0.014s	sys 0.000s
I20260709 12:13:54.161384 25977 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.123472 26157 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.163740 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 805) took 1368 ms. Trace:
I20260709 12:13:54.164645 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.164125 23931 rpcz_store.cc:276] 0709 12:13:52.795097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.796289 (+  1192us) service_pool.cc:224] Handling call
0709 12:13:52.796598 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.844263 (+ 47665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.151019 (+1306756us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54002b6e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:54.163674 (+ 12655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103201152}
W20260709 12:13:54.165813 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.169358 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.177143 23931 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:54.083616 26194 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:54.162943 26176 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:13:54.178524 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:54.142627 26086 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:54.187475 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:54.190028 26018 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:54.165289 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:54.191560 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:54.162117 26192 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.168869 25962 raft_consensus.cc:359] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:54.196076 25962 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:13:54.198902 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.201027 23783 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.195111 26086 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.204876 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:54.205682 23780 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:13:54.206925 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:54.208549 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:54.209199 23780 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:54.197932 26117 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:54.207355 26086 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.166134 26011 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.167479 26098 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.198406 25925 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.190243 26099 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.220924 26185 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 12:13:54.223636 26191 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:13:54.227716 26201 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.228641 26197 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:54.230852 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:54.180966 26196 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:54.236223 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.170480 26151 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.263965 25887 ts_tablet_manager.cc:1434] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.714s	user 0.013s	sys 0.010s
I20260709 12:13:54.287346 26101 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:54.274911 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:54.348377 23926 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
W20260709 12:13:54.285720 26162 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.873s	user 0.006s	sys 0.004s
W20260709 12:13:54.301345 26193 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 12:13:54.278177 26093 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.005s	user 0.005s	sys 0.007s
I20260709 12:13:54.242889 26062 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.263900 26099 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:54.227727 25977 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:54.266156 26201 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.274554 25973 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.255719 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:54.479228 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 379) took 1001 ms. Trace:
W20260709 12:13:54.314293 25887 ts_tablet_manager.cc:1467] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Tablet startup took 763ms
I20260709 12:13:54.439827 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:54.285526 26197 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.284329 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:54.440855 26196 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.476547 26201 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.314203 26098 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.197113 25962 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:54.481858 26099 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.492179 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 123) took 1868 ms. Trace:
I20260709 12:13:54.502146 23785 rpcz_store.cc:276] 0709 12:13:52.623929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.624476 (+   547us) service_pool.cc:224] Handling call
0709 12:13:52.626149 (+  1673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.159751 (+533602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.491913 (+1332162us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400167454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:54.492105 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704779008}
I20260709 12:13:54.494022 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2196) took 1038 ms. Trace:
I20260709 12:13:54.442703 26117 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.503415 23781 rpcz_store.cc:276] 0709 12:13:53.456106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.457805 (+  1699us) service_pool.cc:224] Handling call
0709 12:13:54.493798 (+1035993us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54004263d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:54.493948 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899363712}
I20260709 12:13:54.432904 26192 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:54.482511 26202 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
I20260709 12:13:54.514101 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2351) took 1676 ms. Trace:
I20260709 12:13:54.514549 23860 rpcz_store.cc:276] 0709 12:13:52.837213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.838947 (+  1734us) service_pool.cc:224] Handling call
0709 12:13:52.839285 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.721325 (+882040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.513853 (+792528us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54000703d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:54.514035 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796770048}
I20260709 12:13:54.446669 26093 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.521397 26156 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:13:54.523475 26183 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:54.524097 26183 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:54.506714 26188 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:54.537138 25169 raft_consensus.cc:1217] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:54.525431 26183 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.479792 25170 rpcz_store.cc:276] 0709 12:13:53.479309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.478452 (+999143us) service_pool.cc:224] Handling call
0709 12:13:54.479150 (+   698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.557852 23783 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:13:54.650748 23825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.065s
W20260709 12:13:54.791657 23825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.009s	sys 0.075s
I20260709 12:13:54.792930 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 800) took 2688 ms. Trace:
I20260709 12:13:54.541065 25962 ts_tablet_manager.cc:1434] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.402s	user 0.011s	sys 0.000s
I20260709 12:13:54.794148 23934 rpcz_store.cc:276] 0709 12:13:52.104901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.163116 (+ 58215us) service_pool.cc:224] Handling call
0709 12:13:53.166418 (+1003302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.175538 (+  9120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.789877 (+1614339us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002035d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:54.792863 (+  2986us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2306182400}
I20260709 12:13:54.808247 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.809562 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.807580 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.811070 23852 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:54.700595 26099 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:54.817178 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.807542 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.705278 26197 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:54.787243 26151 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:54.799274 25962 ts_tablet_manager.cc:1467] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Tablet startup took 666ms
I20260709 12:13:54.791163 26196 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.813819 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.812225 26183 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.814571 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.814427 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:54.817922 23860 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:13:54.828287 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.567890 26157 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.820168 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.832325 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.823894 26099 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.511929 26117 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.820832 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.823297 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.834944 23852 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:54.770229 26201 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.812191 26093 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:54.827605 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.829429 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.840107 23860 raft_consensus.cc:2410] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:13:54.819450 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.841614 23858 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:13:54.842029 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:54.828174 26062 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.849507 26062 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.855701 26107 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:54.828032 26192 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.843624 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:54.845335 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.858090 23859 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:54.827476 26130 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:54.827841 26101 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:54.859894 26130 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.865643 26192 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.833918 26098 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:54.836743 26184 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:54.837363 26196 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:54.844666 26080 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.006s
I20260709 12:13:54.863000 26093 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:54.881690 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:54.885052 23858 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:54.883050 23860 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:13:54.883175 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:54.886770 23855 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:54.840132 26099 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.896727 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:54.877803 26158 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:54.871192 23856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.016s	sys 0.025s
W20260709 12:13:54.889283 26180 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.819s	user 0.003s	sys 0.000s
W20260709 12:13:54.905236 23778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.002s	sys 0.017s
W20260709 12:13:54.910791 23778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.002s	sys 0.017s
W20260709 12:13:54.905972 23856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.016s	sys 0.026s
W20260709 12:13:54.899863 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.004s	sys 0.016s
I20260709 12:13:54.904654 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 589) took 1466 ms. Trace:
I20260709 12:13:54.881430 26184 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:54.914357 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.004s	sys 0.016s
W20260709 12:13:54.915752 23824 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.68:43683 timed out after 1.751s (SENT) [suppressed 30 similar messages]
I20260709 12:13:54.914414 25175 rpcz_store.cc:276] 0709 12:13:53.438065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.438182 (+   117us) service_pool.cc:224] Handling call
0709 12:13:54.896540 (+1458358us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:13:54.904576 (+  8036us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3651328}
I20260709 12:13:54.872628 26101 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.930546 26192 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.931192 26192 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:54.481863 25887 ts_tablet_manager.cc:1469] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Trace:
0709 12:13:43.050286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.050403 (+   117us) service_pool.cc:224] Handling call
0709 12:13:44.076632 (+1026229us) ts_tablet_manager.cc:934] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
0709 12:13:48.190771 (+4114139us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:48.990124 (+799353us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:49.021289 (+ 31165us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:49.354930 (+333641us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:49.411084 (+ 56154us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:49.451344 (+ 40260us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:49.451469 (+   125us) inbound_call.cc:177] Queueing success response
0709 12:13:50.668967 (+1217498us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:53.550190 (+2881223us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:53.735990 (+185800us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:53.747903 (+ 11913us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":1989,"mutex_wait_us":21679,"raft.queue_time_us":6975,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":12115,"tablet-copy.queue_time_us":920463,"tablet-download-pool.queue_time_us":171,"tablet-download-pool.run_cpu_time_us":1265,"tablet-download-pool.run_wall_time_us":344126,"thread_start_us":1716375,"threads_started":2}
I20260709 12:13:54.943624 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.946316 23783 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:54.951730 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:54.952457 23776 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.879364 26098 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.881273 26196 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:54.910997 26157 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:54.883036 26080 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:54.894510 26093 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:54.976756 23925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.010s	sys 0.009s
W20260709 12:13:54.980038 23925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.011s	sys 0.009s
W20260709 12:13:54.977208 23931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.000s	sys 0.020s
W20260709 12:13:54.981526 23931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.000s	sys 0.020s
W20260709 12:13:54.979247 23928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.018s	sys 0.034s
W20260709 12:13:54.983762 23928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.019s	sys 0.034s
I20260709 12:13:54.944386 26062 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:55.004562 26194 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.163137 26117 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:55.067344 26036 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.010s	sys 0.018s
I20260709 12:13:55.159857 26098 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:55.156486 26094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.011s	sys 0.021s
W20260709 12:13:54.992614 23857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.000s	sys 0.051s
I20260709 12:13:55.013681 23783 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
W20260709 12:13:55.164984 23857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.000s	sys 0.062s
W20260709 12:13:55.068631 23777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.010s	sys 0.013s
W20260709 12:13:55.165911 23777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.011s	sys 0.018s
I20260709 12:13:55.165678 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2102) took 1979 ms. Trace:
W20260709 12:13:55.164186 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 130) took 2340 ms (client timeout 2214 ms). Trace:
I20260709 12:13:55.166725 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:55.166445 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:55.079269 23898 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.011s	sys 0.022s
W20260709 12:13:55.169848 23898 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.011s	sys 0.022s
W20260709 12:13:55.170279 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 920604us
I20260709 12:13:55.169051 23783 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:55.171252 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.172219 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.081261 23776 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:55.166918 23933 rpcz_store.cc:276] 0709 12:13:53.185954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.186080 (+   126us) service_pool.cc:224] Handling call
0709 12:13:54.151305 (+965225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.161065 (+  9760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.165414 (+1004349us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54002b6e50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:55.165615 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219069184}
I20260709 12:13:55.174283 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:55.167954 23929 rpcz_store.cc:269] 0709 12:13:52.824129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.824305 (+   176us) service_pool.cc:224] Handling call
0709 12:13:55.163933 (+2339628us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54002a3d54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:55.164084 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171192064}
W20260709 12:13:55.163997 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 78783.8us
W20260709 12:13:55.165448 26036 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.013s	sys 0.022s
I20260709 12:13:55.163776 26093 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.018986 26192 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:55.164623 26094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.013s	sys 0.022s
I20260709 12:13:55.225723 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2114) took 1178 ms. Trace:
I20260709 12:13:54.998057 23855 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
W20260709 12:13:55.227183 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.004s	sys 0.019s
I20260709 12:13:54.996198 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 127) took 2088 ms. Trace:
W20260709 12:13:55.227579 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.004s	sys 0.019s
W20260709 12:13:55.086654 23780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.005s	sys 0.014s
W20260709 12:13:55.228287 23784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.006s	sys 0.018s
I20260709 12:13:55.016150 26201 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:55.228631 23780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.009s	sys 0.015s
I20260709 12:13:55.227843 23782 rpcz_store.cc:276] 0709 12:13:52.908262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.908411 (+   149us) service_pool.cc:224] Handling call
0709 12:13:52.908737 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.176116 (+267379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.176151 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:53.176195 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:53.176232 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.995904 (+1819672us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54003ba954. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:54.995958 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:54.996126 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":614460160}
W20260709 12:13:55.223982 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44470.8us
W20260709 12:13:54.824031 25962 ts_tablet_manager.cc:1469] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Trace:
0709 12:13:46.753956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.756394 (+  2438us) service_pool.cc:224] Handling call
0709 12:13:47.212710 (+456316us) ts_tablet_manager.cc:934] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
0709 12:13:48.541950 (+1329240us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:49.198460 (+656510us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:49.198575 (+   115us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:49.632980 (+434405us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:49.663426 (+ 30446us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:49.753058 (+ 89632us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:49.762995 (+  9937us) inbound_call.cc:177] Queueing success response
0709 12:13:50.858663 (+1095668us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:54.141545 (+3282882us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:54.155250 (+ 13705us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:54.156308 (+  1058us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":23934,"raft.queue_time_us":7987,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":2841,"tablet-copy.queue_time_us":319111,"tablet-download-pool.queue_time_us":19408,"tablet-download-pool.run_cpu_time_us":1300,"tablet-download-pool.run_wall_time_us":43826,"thread_start_us":701686,"threads_started":2}
W20260709 12:13:55.114008 23785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.003s	sys 0.019s
I20260709 12:13:55.178900 26158 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:55.228688 23784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.006s	sys 0.018s
W20260709 12:13:55.180233 26154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.014s	sys 0.018s
W20260709 12:13:55.104496 26119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.007s	sys 0.015s
I20260709 12:13:55.234932 23783 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:55.023743 26052 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:55.180438 25967 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:13:55.233073 23785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.004s	sys 0.020s
W20260709 12:13:55.238440 26154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.015s	sys 0.018s
I20260709 12:13:55.020442 26087 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:55.232450 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.226171 23928 rpcz_store.cc:276] 0709 12:13:54.047147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.047467 (+   320us) service_pool.cc:224] Handling call
0709 12:13:54.047776 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.047986 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:54.057700 (+  9714us) log.cc:844] Serialized 14 byte log entry
0709 12:13:55.225363 (+1167663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.225418 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:55.225476 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.225540 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.225665 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12005,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"thread_start_us":927287,"threads_started":1}
W20260709 12:13:55.245760 26119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.308s	user 0.010s	sys 0.015s
W20260709 12:13:55.249830 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.011s	sys 0.032s
I20260709 12:13:55.146970 26208 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:13:55.236207 26196 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:55.251524 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.253444 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.226s	user 0.011s	sys 0.032s
W20260709 12:13:55.257172 23899 connection.cc:256] RPC call timeout handler was delayed by -1.8023003099999642s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:13:55.248373 26130 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.247023 26052 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:55.258155 23899 connection.cc:256] RPC call timeout handler was delayed by -1.8005003099999612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 12:13:55.247357 26099 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.252415 26087 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:55.253670 26184 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:13:55.269951 26179 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.035s	sys 0.040s
W20260709 12:13:55.270457 26179 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.035s	sys 0.040s
I20260709 12:13:55.270969 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:55.271580 25176 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:55.261300 23899 connection.cc:256] RPC call timeout handler was delayed by -1.7384003099998608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
I20260709 12:13:55.251756 25967 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:55.274049 25967 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.275035 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:13:55.277192 23899 connection.cc:256] RPC call timeout handler was delayed by -1.724900309999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 12:13:55.277546 23899 connection.cc:256] RPC call timeout handler was delayed by -1.7186003099998288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 12:13:55.277848 23899 connection.cc:256] RPC call timeout handler was delayed by -1.3406003099998998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
W20260709 12:13:55.278127 23899 connection.cc:256] RPC call timeout handler was delayed by -1.2947003100000529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 12:13:55.278298 23930 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
W20260709 12:13:55.279229 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.279927 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.281667 25175 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:55.284919 25176 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:13:55.286156 23933 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:55.286571 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 580) took 1886 ms. Trace:
W20260709 12:13:55.287803 23779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.010s	sys 0.053s
W20260709 12:13:55.288499 23779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.010s	sys 0.053s
I20260709 12:13:55.293220 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:55.295140 26092 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:55.296276 26092 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:55.296176 25168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 581) took 1895 ms. Trace:
I20260709 12:13:55.296909 26092 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:55.251119 26101 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.296159 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.296952 25168 rpcz_store.cc:276] 0709 12:13:53.400760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.408528 (+  7768us) service_pool.cc:224] Handling call
0709 12:13:53.432845 (+ 24317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.750556 (+317711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.295944 (+1545388us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54001fac54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:55.296116 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267268864}
I20260709 12:13:55.286938 25171 rpcz_store.cc:276] 0709 12:13:53.400495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.404343 (+  3848us) service_pool.cc:224] Handling call
0709 12:13:53.407297 (+  2954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.010322 (+603025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.286325 (+1276003us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b5400476654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:13:55.286511 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192936320}
I20260709 12:13:55.299660 26119 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:55.300631 26119 consensus_queue.cc:1243] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:55.301200 26107 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:55.301674 26107 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:55.302696 26107 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.297674 26182 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:55.303885 26182 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:55.304311 26107 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:55.304853 26182 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.306486 26182 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.310235 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.311240 23776 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:55.312698 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.313324 23928 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:55.314455 26119 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:55.314337 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.317142 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.320051 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.321558 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.322180 23928 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:55.322362 23932 raft_consensus.cc:2410] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:13:55.322831 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.320902 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.324164 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.325266 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.325883 23927 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:55.314623 25987 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:55.324098 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:55.327032 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.327474 23931 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:55.327885 26095 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:55.328352 26095 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:55.328740 26095 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:55.332219 26119 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.332788 26119 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:55.322466 23783 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:55.335021 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.331254 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:55.335739 25171 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:55.336551 26095 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:55.337843 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.338430 23931 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
I20260709 12:13:55.338820 26095 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:55.327150 25987 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:55.340528 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.342088 26167 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:55.341590 25987 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.342613 26167 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:55.344233 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:55.344031 26167 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.346813 25987 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:55.338722 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 136) took 2093 ms (client timeout 1875 ms). Trace:
I20260709 12:13:55.355194 26107 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:55.353754 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.345932 26119 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:55.346501 26104 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:55.362099 26149 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:55.363516 26036 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:55.363966 26036 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:55.371321 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.011s
W20260709 12:13:55.371847 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.013s	sys 0.011s
I20260709 12:13:55.372298 26092 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:55.377607 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 513370us
I20260709 12:13:55.263890 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d6d3d4f634284532bbf6ad7d4ac19214 received from {username='slave'} at 127.0.0.1:40134
W20260709 12:13:55.378476 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 462295us
I20260709 12:13:55.393186 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:55.393805 23927 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:55.351639 26087 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.394399 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:55.394971 25176 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:55.395632 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 76779.4us
I20260709 12:13:55.398151 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.399978 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.405745 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:55.349035 23930 rpcz_store.cc:269] 0709 12:13:53.245475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.245894 (+   419us) service_pool.cc:224] Handling call
0709 12:13:55.338226 (+2092332us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54002b5554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:55.338486 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4566107392}
I20260709 12:13:55.364903 26036 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.408334 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.410593 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:55.412158 25176 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:55.414125 23927 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:55.419180 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:55.425534 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 169) took 1459 ms. Trace:
I20260709 12:13:55.425314 26204 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:55.425819 26185 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.426340 26185 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:55.429152 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.430138 26036 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:55.431783 26184 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:55.430016 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:55.432260 26184 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:55.430883 23857 rpcz_store.cc:276] 0709 12:13:53.965841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.965977 (+   136us) service_pool.cc:224] Handling call
0709 12:13:53.966327 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.994909 (+ 28582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.994942 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:53.995390 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.165525 (+1170135us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.165565 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.165631 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.425287 (+259656us) raft_consensus.cc:1685] finished
0709 12:13:55.425308 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.425471 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28154,"raft.queue_time_us":33874,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":7091,"thread_start_us":1132168,"threads_started":1,"wal-append.queue_time_us":1385558}
I20260709 12:13:55.432677 26184 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.433842 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:55.435012 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:55.435954 23857 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:55.436182 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:55.437589 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:55.438009 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2122) took 1156 ms. Trace:
I20260709 12:13:55.435897 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:55.438365 23925 rpcz_store.cc:276] 0709 12:13:54.281711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.281975 (+   264us) service_pool.cc:224] Handling call
0709 12:13:54.282489 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.282857 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.282886 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.286739 (+  3853us) log.cc:844] Serialized 26 byte log entry
0709 12:13:54.981113 (+694374us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:54.981155 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.981220 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:55.437823 (+456603us) raft_consensus.cc:1685] finished
0709 12:13:55.437841 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.437954 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":10766,"thread_start_us":693873,"threads_started":1,"wal-append.queue_time_us":1149217}
I20260709 12:13:55.380472 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.441200 26101 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:55.442664 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 62280.5us
I20260709 12:13:55.428649 26185 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:55.378594 26080 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.448024 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.395655 26052 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.450176 26052 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:55.452416 26087 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:55.454103 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.455148 26174 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:55.435770 26201 raft_consensus.cc:2764] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:13:55.475530 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.458966 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.350904 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.455574 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:55.455993 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.457171 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.458930 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.409998 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:55.464731 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.467689 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.468616 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:55.470402 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.027s	sys 0.032s
I20260709 12:13:55.460238 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.471855 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:55.497404 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 146538us
I20260709 12:13:55.498359 23925 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:55.484887 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.499375 23934 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:55.402326 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d6d3d4f634284532bbf6ad7d4ac19214 on tablet d6d3d4f634284532bbf6ad7d4ac19214 with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:13:55.500586 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45104.8us
W20260709 12:13:55.497630 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.028s	sys 0.032s
W20260709 12:13:55.503645 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.507987 25119 connection.cc:256] RPC call timeout handler was delayed by -1.1582399620001524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 12:13:55.503563 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.506058 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:55.498684 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.517591 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:55.518591 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.528322 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:55.529718 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.529073 23926 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.529080 25119 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.534229 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:55.562002 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:55.543169 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:55.501579 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:55.502323 26080 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:55.502028 26204 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:55.506723 26194 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:55.584581 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.585599 23931 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:55.538679 26099 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:55.564378 26175 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.022s	sys 0.019s
I20260709 12:13:55.541358 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:55.538864 26196 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.502940 25173 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:13:55.561904 25843 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:55.605098 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 101601us
I20260709 12:13:55.529191 26101 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:55.538616 26192 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:55.663692 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.584307 26080 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:55.571504 26174 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.562000 26068 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:55.562244 26201 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:55.740829 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.544999 26087 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:55.742233 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.563026 26104 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
W20260709 12:13:55.738924 25173 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 381) took 1719 ms (client timeout 1781 ms). Trace:
W20260709 12:13:55.791776 25173 rpcz_store.cc:269] 0709 12:13:54.019958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.458353 (+1438395us) service_pool.cc:224] Handling call
0709 12:13:55.665896 (+207543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15744}
I20260709 12:13:55.792161 26104 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:55.792601 26104 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:55.792965 26104 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:55.764079 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 222748us
I20260709 12:13:55.772923 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.607936 26099 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:55.776177 23926 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:13:55.608234 26204 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:55.809104 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.798481 23933 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:55.813752 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.814430 23933 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:55.758097 26101 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.666003 26215 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:55.763458 26161 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.768244 26080 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:55.815282 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:55.817744 23926 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:55.742820 25843 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:55.758419 26192 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:55.820742 23933 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:13:55.824671 25894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.005s	sys 0.008s
I20260709 12:13:55.811475 26099 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:55.819454 26101 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:55.826205 25843 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.829768 26192 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:55.800794 26201 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.795893 26068 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:55.833145 26158 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.013s
I20260709 12:13:55.860869 26046 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:13:55.850399 25169 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.003s	sys 0.011s
W20260709 12:13:55.881111 26167 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.013s
W20260709 12:13:55.890424 25894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.009s
W20260709 12:13:55.896294 26158 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.001s	sys 0.013s
W20260709 12:13:55.899567 25169 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.003s	sys 0.011s
W20260709 12:13:55.901024 26167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.003s	sys 0.013s
I20260709 12:13:55.897998 26046 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:13:55.898729 26117 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.024s	sys 0.021s
I20260709 12:13:55.908579 26158 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:55.900182 26204 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:55.916772 26117 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.025s	sys 0.021s
W20260709 12:13:55.604491 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70328.2us
I20260709 12:13:55.743780 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:55.784467 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.130936 26239 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:56.098373 23929 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.013s	sys 0.000s
I20260709 12:13:55.995227 26080 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:56.081048 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.097502 23930 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.006s	sys 0.074s
W20260709 12:13:56.083714 23777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.010s	sys 0.014s
W20260709 12:13:56.077847 23780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.003s	sys 0.007s
W20260709 12:13:55.917464 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 379582us
I20260709 12:13:56.009331 26068 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:55.994091 26184 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.003s
I20260709 12:13:56.136591 26184 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.138367 23859 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:56.083415 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:56.140268 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57192.8us
I20260709 12:13:56.142052 23781 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:56.083847 23784 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.007s	sys 0.003s
W20260709 12:13:56.144379 23784 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.008s	sys 0.003s
I20260709 12:13:56.144584 23782 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:56.145818 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2143) took 2070 ms. Trace:
I20260709 12:13:56.147125 26087 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:56.147766 26087 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.148315 26087 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:56.147457 23930 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.006s	sys 0.074s
W20260709 12:13:56.147687 23777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.010s	sys 0.015s
I20260709 12:13:56.152092 25925 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:56.147070 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66559.8us
I20260709 12:13:56.156821 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.157871 23780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.004s	sys 0.008s
I20260709 12:13:56.145887 26080 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:13:56.146325 23784 rpcz_store.cc:276] 0709 12:13:54.077654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.305190 (+227536us) service_pool.cc:224] Handling call
0709 12:13:54.305568 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.307170 (+  1602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.307202 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.313250 (+  6048us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.237286 (+924036us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.144892 (+907606us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.145375 (+   483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.145497 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.145555 (+    58us) raft_consensus.cc:1685] finished
0709 12:13:56.145566 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.145745 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10084,"prepare.queue_time_us":695010,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":3111,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":403,"thread_start_us":1812368,"threads_started":2,"wal-append.queue_time_us":1482694}
I20260709 12:13:56.161092 26196 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:13:56.118151 23783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.002s	sys 0.009s
W20260709 12:13:56.097879 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.002s	sys 0.017s
W20260709 12:13:56.076290 23778 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.005s	sys 0.017s
I20260709 12:13:55.809175 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:56.103973 25113 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.011s
I20260709 12:13:55.999965 26201 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.027165 26192 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:56.104550 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.006s	sys 0.005s
I20260709 12:13:56.010774 26101 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:56.098044 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 298983us
I20260709 12:13:55.957368 26046 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.077536 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2145) took 1990 ms. Trace:
I20260709 12:13:56.021404 25843 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:56.127662 26117 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.999433 26104 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:55.986696 26099 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.009889 26158 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:55.951368 26183 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.123s	user 0.011s	sys 0.004s
I20260709 12:13:56.216317 25843 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:56.131490 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 387619us
I20260709 12:13:56.132400 26174 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:56.132740 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 348224us
I20260709 12:13:56.218008 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:56.220459 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:56.223102 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.929612 25967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.009s	sys 0.005s
W20260709 12:13:56.133540 23929 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.014s	sys 0.001s
W20260709 12:13:56.224009 25967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.010s	sys 0.006s
I20260709 12:13:56.163329 26196 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:56.171296 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.003s	sys 0.017s
W20260709 12:13:56.175864 23783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.010s
W20260709 12:13:56.205582 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.006s
W20260709 12:13:56.205951 23778 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.006s	sys 0.017s
W20260709 12:13:56.207299 25113 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.012s	sys 0.011s
I20260709 12:13:56.143399 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:56.209961 26117 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.211174 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:56.201628 23779 rpcz_store.cc:276] 0709 12:13:54.096507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.492785 (+396278us) service_pool.cc:224] Handling call
0709 12:13:54.493191 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.493600 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.493629 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.504746 (+ 11117us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.289187 (+784441us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:55.289249 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.289370 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.076551 (+787181us) raft_consensus.cc:1685] finished
0709 12:13:56.076626 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.077137 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6068,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":7820,"thread_start_us":722708,"threads_started":1,"wal-append.queue_time_us":1013465}
I20260709 12:13:56.212237 26104 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.213936 26101 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.214506 26099 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.219180 26192 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:56.237893 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 736552us
I20260709 12:13:56.241279 26068 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:56.241998 25170 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
W20260709 12:13:56.235210 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 692059us
I20260709 12:13:56.231338 26184 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.248202 26204 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:56.251356 26117 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.257752 26196 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.261098 26188 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:56.263641 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52767.5us
W20260709 12:13:56.264537 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.266134 26099 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:56.266665 26099 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.267088 26099 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.254735 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2144) took 2173 ms. Trace:
I20260709 12:13:56.263581 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2142) took 2188 ms. Trace:
I20260709 12:13:56.280256 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:56.276582 25113 leader_election.cc:336] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.609s (SENT)
I20260709 12:13:56.289425 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.292397 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.293382 26149 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:56.294494 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.297209 26225 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:56.298156 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2141) took 2234 ms. Trace:
I20260709 12:13:56.298596 23778 rpcz_store.cc:276] 0709 12:13:54.063912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.076635 (+ 12723us) service_pool.cc:224] Handling call
0709 12:13:54.097354 (+ 20719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.911215 (+813861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.911247 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.926916 (+ 15669us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.382020 (+455104us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.240153 (+858133us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.283481 (+ 43328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.283624 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.283675 (+    51us) raft_consensus.cc:1685] finished
0709 12:13:56.283686 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.298083 (+ 14397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22031,"prepare.queue_time_us":562135,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":73352,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":241,"spinlock_wait_cycles":23296,"thread_start_us":2010021,"threads_started":3,"wal-append.queue_time_us":824184}
I20260709 12:13:56.301254 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:56.301586 26222 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.505s	user 0.007s	sys 0.011s
I20260709 12:13:56.302193 25172 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:56.303634 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2146) took 2189 ms. Trace:
I20260709 12:13:56.304077 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:56.304884 26065 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.304065 23785 rpcz_store.cc:276] 0709 12:13:54.114055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.505425 (+391370us) service_pool.cc:224] Handling call
0709 12:13:54.505849 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.506178 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.506204 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.512223 (+  6019us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.248493 (+736270us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:55.248543 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.248967 (+   424us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.262350 (+1013383us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:56.303063 (+ 40713us) raft_consensus.cc:1685] finished
0709 12:13:56.303085 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.303412 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3822,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2841,"spinlock_wait_cycles":20520192,"thread_start_us":683942,"threads_started":1,"wal-append.queue_time_us":1050101}
W20260709 12:13:56.312587 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 92431.4us
I20260709 12:13:56.313748 26052 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:56.314234 26052 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:56.315796 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.276280 26101 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:56.322506 26046 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.323189 26188 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.327054 26239 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:56.327557 26239 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.328008 26239 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:56.318615 26052 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.294916 26149 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:56.333762 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-de3c70defd864769b26b52b7f36548cd received from {username='slave'} at 127.0.0.1:46448
I20260709 12:13:56.334506 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-de3c70defd864769b26b52b7f36548cd on tablet de3c70defd864769b26b52b7f36548cd with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:56.335987 25978 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.337224 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.338840 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2124) took 1057 ms. Trace:
I20260709 12:13:56.339279 23929 rpcz_store.cc:276] 0709 12:13:55.281020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.282794 (+  1774us) service_pool.cc:224] Handling call
0709 12:13:55.283105 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.283302 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:56.224308 (+941006us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.277196 (+ 52888us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.281179 (+  3983us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:56.321290 (+ 40111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.338505 (+ 17215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.338729 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":374,"prepare.queue_time_us":674782,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":136,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":1574,"thread_start_us":989778,"threads_started":2,"wal-append.queue_time_us":100317}
I20260709 12:13:56.340363 26175 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:56.341131 26175 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.347606 25843 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:56.350123 25978 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:56.350590 25978 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:56.351742 25978 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.268224 23780 rpcz_store.cc:276] 0709 12:13:54.081101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.314166 (+233065us) service_pool.cc:224] Handling call
0709 12:13:54.445025 (+130859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.480180 (+ 35155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.480216 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.485269 (+  5053us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.231350 (+746081us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.158594 (+927244us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.239230 (+ 80636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.246265 (+  7035us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.246314 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:56.246324 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.254668 (+  8344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4931,"prepare.queue_time_us":693391,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":108820,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":8175,"thread_start_us":1692671,"threads_started":3,"wal-append.queue_time_us":905204}
I20260709 12:13:56.368911 25169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 595) took 1075 ms. Trace:
W20260709 12:13:56.331314 26214 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.563s	user 0.001s	sys 0.014s
W20260709 12:13:56.389529 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 246334us
I20260709 12:13:56.384177 26052 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:56.398211 25978 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:56.398730 23859 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:56.404386 23683 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:56.406087 26183 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.407660 26080 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:56.409283 26175 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:13:56.397037 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 240410us
W20260709 12:13:56.412899 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 132857us
I20260709 12:13:56.276630 23777 rpcz_store.cc:276] 0709 12:13:54.075379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.165674 (+ 90295us) service_pool.cc:224] Handling call
0709 12:13:54.170986 (+  5312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.171348 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.171378 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.179741 (+  8363us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.166269 (+986528us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:56.150841 (+984572us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.245433 (+ 94592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.247174 (+  1741us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.255371 (+  8197us) raft_consensus.cc:1685] finished
0709 12:13:56.255380 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.263408 (+  8028us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19473,"prepare.queue_time_us":741735,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1666,"raft.queue_time_us":118243,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":15598,"spinlock_wait_cycles":1408,"thread_start_us":1951962,"threads_started":3,"wal-append.queue_time_us":1263446}
I20260709 12:13:56.430192 23780 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:56.444604 26161 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:56.447100 23783 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:56.447412 26175 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:56.450798 26175 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.453606 26175 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.436210 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:56.469841 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 126012us
W20260709 12:13:56.475116 25174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 386) took 2361 ms (client timeout 1932 ms). Trace:
W20260709 12:13:56.479775 25174 rpcz_store.cc:269] 0709 12:13:54.113163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.459849 (+1346686us) service_pool.cc:224] Handling call
0709 12:13:56.474664 (+1014815us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b540051c554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:13:56.474898 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238766464}
I20260709 12:13:56.476454 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:56.477314 25113 leader_election.cc:336] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 2.061s (SENT)
I20260709 12:13:56.482649 25113 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.484066 25113 leader_election.cc:336] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 2.104s (SENT)
I20260709 12:13:56.484515 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.484413 26104 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:56.485785 26104 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:56.488287 26087 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.490684 26087 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.478614 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 274) took 5263 ms. Trace:
I20260709 12:13:56.430884 26080 consensus_queue.cc:1243] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:56.491318 23680 rpcz_store.cc:276] 0709 12:13:51.214633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.214838 (+   205us) service_pool.cc:224] Handling call
0709 12:13:56.478342 (+5263504us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:53.971053 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:54.039194 (+ 68141us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:54.043225 (+  4031us) write_op.cc:435] Acquired schema lock
0709 12:13:54.050536 (+  7311us) tablet.cc:662] Decoding operations
0709 12:13:54.133999 (+ 83463us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:54.134087 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:54.134130 (+    43us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:13:54.263730 (+129600us) tablet.cc:701] Row locks acquired
0709 12:13:54.266526 (+  2796us) write_op.cc:260] PREPARE: finished
0709 12:13:54.294645 (+ 28119us) write_op.cc:270] Start()
0709 12:13:54.299695 (+  5050us) write_op.cc:276] Timestamp: P: 1783599234287327 usec, L: 0
0709 12:13:54.303334 (+  3639us) op_driver.cc:348] REPLICATION: starting
0709 12:13:54.510594 (+207260us) log.cc:844] Serialized 4052 byte log entry
0709 12:13:56.005158 (+1494564us) op_driver.cc:464] REPLICATION: finished
0709 12:13:56.034626 (+ 29468us) write_op.cc:301] APPLY: starting
0709 12:13:56.034739 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:56.035089 (+   350us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:56.035280 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:56.036431 (+  1151us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:56.036612 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:13:56.036641 (+    29us) write_op.cc:312] APPLY: finished
0709 12:13:56.113950 (+ 77309us) log.cc:844] Serialized 77 byte log entry
0709 12:13:56.114429 (+   479us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:56.128154 (+ 13725us) write_op.cc:454] Released schema lock
0709 12:13:56.128283 (+   129us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:56.211138 (+ 82855us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:56.211169 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":67,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":29177,"apply.run_cpu_time_us":6853,"apply.run_wall_time_us":174892,"num_ops":10,"prepare.queue_time_us":765674,"prepare.run_cpu_time_us":86373,"prepare.run_wall_time_us":1330703,"raft.queue_time_us":410953,"raft.run_cpu_time_us":10581,"raft.run_wall_time_us":29333,"replication_time_us":1701675,"thread_start_us":1507835,"threads_started":4,"wal-append.queue_time_us":1078669}]]}
I20260709 12:13:56.497346 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.436764 26183 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.498185 26186 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.499221 26186 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:56.500417 26183 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:56.500957 23783 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:56.501408 26186 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:56.503083 23899 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.504012 26186 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:56.505541 26161 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:56.506978 26161 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:56.471771 26188 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:13:56.508526 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:56.508937 26146 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:56.509402 26146 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:56.369478 25169 rpcz_store.cc:276] 0709 12:13:55.293405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.294671 (+  1266us) service_pool.cc:224] Handling call
0709 12:13:55.295019 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.295238 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:55.917051 (+621813us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.312451 (+395400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.368300 (+ 55849us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:56.368412 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.368516 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.368719 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17346,"prepare.queue_time_us":620084,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":36998,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":5293,"thread_start_us":697224,"threads_started":2,"wal-append.queue_time_us":173418}
I20260709 12:13:56.509521 26175 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:56.526270 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.508441 26161 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:13:56.530843 23855 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:56.532711 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.532647 26161 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:56.533164 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:56.535137 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.535876 26157 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:56.532593 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:56.539746 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.541877 26188 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:56.530292 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.539664 26161 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:56.552769 26175 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:56.555766 26186 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:56.556349 26186 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:56.554795 23780 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:56.556758 26186 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.562321 23683 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 4, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 4 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:56.564180 26094 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:56.565757 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:56.525666 26239 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.553023 26157 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.571183 26157 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.556784 26175 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.560173 26188 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:56.568925 26094 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:56.539554 26146 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:56.589726 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54697.5us
I20260709 12:13:56.586531 26259 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.585445 26094 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:56.594677 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 123712us
W20260709 12:13:56.603132 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 123151us
I20260709 12:13:56.609055 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.613950 26152 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:56.614580 26152 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.615012 26152 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:56.621933 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.621817 26201 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:56.624778 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:56.624502 26188 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.628571 26161 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.630048 26161 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:56.634022 23780 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:56.635912 26087 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:56.636351 26087 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:56.636718 26087 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:56.635854 26183 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:56.639884 25925 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:56.640334 25925 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:56.640822 25925 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:56.641997 26099 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.640796 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.669098 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.666375 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.664593 25843 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:56.685225 25843 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:56.686642 26092 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:56.688941 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.689648 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.666824 26259 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:56.694356 26186 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.698145 23784 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:56.698700 26068 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:56.701521 26099 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.674247 26146 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:56.709703 26092 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:56.712029 25843 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:56.714658 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 112287us
I20260709 12:13:56.714502 26157 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.715088 25925 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.721232 26239 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.751466 23785 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:56.744524 26080 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:56.755920 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 115278us
I20260709 12:13:56.746132 26184 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:56.762483 26036 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.763108 26036 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.765348 26036 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.771410 26092 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.773967 25987 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:13:56.775346 25987 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:56.775660 25925 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.758697 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:56.780207 25168 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:56.765236 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 152444us
I20260709 12:13:56.775779 25987 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:56.747687 26087 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.785347 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.785609 25987 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.790365 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:56.790997 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.791564 23778 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.783066 26157 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:56.786372 26117 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:56.797523 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.837491 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:56.839418 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.798483 26117 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:56.843116 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47463.3us
I20260709 12:13:56.847040 26186 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:56.852051 26266 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:56.854876 25843 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.798285 26080 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:56.863466 26226 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:56.864399 26271 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:13:56.864781 26226 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:56.865417 26226 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:56.860683 26186 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:13:56.860682 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:13:56.865653 26273 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:56.860742 26266 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:56.868090 26266 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:56.859357 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68478.6us
I20260709 12:13:56.871623 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.872354 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.872859 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.874593 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.875204 23856 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.880982 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.867813 26184 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:56.873252 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.805513 26068 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.830756 25987 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:56.805742 26099 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:56.892401 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 60687.6us
I20260709 12:13:56.873919 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.875681 26117 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:56.904213 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.907444 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.915192 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:56.916652 23930 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:56.874002 26080 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 12:13:56.922452 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62374.3us
I20260709 12:13:56.927970 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:56.931751 23929 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:13:56.920089 26259 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:56.948067 26226 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.949321 26186 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.949558 26226 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:56.950073 26186 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:56.952539 26275 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:13:56.944094 26080 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:56.925093 26239 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.945813 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.939896 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.915009 26068 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.959529 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:56.957593 26080 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:56.967909 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.963950 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.933495 26152 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:56.982754 26259 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.963613 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.957617 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:56.985165 25978 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:56.963508 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:56.985770 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.986526 23853 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:56.986845 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.982703 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:56.969201 23855 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:56.987462 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:56.979512 26239 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:56.982184 23929 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:56.993497 25169 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEARNER]: Advancing to term 6
I20260709 12:13:56.991966 25173 raft_consensus.cc:2368] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:56.995986 23932 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:56.997363 26186 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:56.997625 26152 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.998765 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:56.997839 23929 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:57.003104 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.004360 26052 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:57.007019 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:56.992044 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60458.3us
I20260709 12:13:57.014230 26259 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:57.016299 26068 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:56.981092 26080 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.011427 26087 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.018108 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.026127 23778 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.018015 26080 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:56.998214 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:57.037774 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.029193 26087 raft_consensus.cc:2764] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:57.041774 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:57.044586 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.031042 26239 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.035646 26146 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.053742 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.042563 25173 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:57.057047 26161 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:57.059700 25925 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:57.071478 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:57.044314 26266 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.069428 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.069110 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.073969 23856 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.070807 25925 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.068606 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:57.104751 25170 raft_consensus.cc:2410] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:57.072340 23854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:57.108016 23854 raft_consensus.cc:606] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:57.079872 26087 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:57.082450 23857 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.109333 26087 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:57.109828 26087 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.110839 26266 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:57.078984 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:13:57.116098 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.116688 23857 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:57.119716 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.116734 23860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:57.121623 23860 raft_consensus.cc:606] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:57.122045 23860 raft_consensus.cc:614] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:57.132997 23930 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:57.144169 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.144533 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.144946 23927 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.145098 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.084231 26117 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.146608 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.147176 26117 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:57.148134 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.148343 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.090891 26281 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.148909 23925 raft_consensus.cc:2376] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3. Re-sending same reply.
I20260709 12:13:57.149027 26281 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.151154 26281 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:57.091362 26188 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.151960 26188 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.152405 23926 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:57.152664 23933 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:57.153734 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.083029 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:57.064611 26146 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.074779 26161 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:57.133699 23860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:57.060146 26094 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:57.154114 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.099718 25925 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:57.154202 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.156478 23860 raft_consensus.cc:606] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.154294 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.156376 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.160507 23857 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:13:57.161270 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 78315.5us
I20260709 12:13:57.172006 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.083483 26276 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:57.184624 23852 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.154033 26188 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:57.188547 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.171938 26259 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.190647 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.195951 23933 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:57.188139 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.204167 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.176295 25168 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:57.184698 26276 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.209154 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.210904 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.190582 26225 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:57.230332 23855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:13:57.231009 23855 raft_consensus.cc:606] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:57.216490 26099 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:57.224798 26146 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:57.226666 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.206629 26259 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.172530 26087 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.218272 26161 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.232136 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.248771 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.265811 23778 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.217715 26276 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.230701 26225 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.265795 23860 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:57.285992 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 108679us
I20260709 12:13:57.288371 23931 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.273960 23859 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.280457 26094 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.272982 25925 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.298885 23852 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:13:57.325553 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 112204us
I20260709 12:13:57.326454 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.327134 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:57.328625 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 58818.7us
W20260709 12:13:57.331576 25113 leader_election.cc:336] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.875s (SENT)
I20260709 12:13:57.332280 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:57.367012 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.325569 26225 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:57.380698 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 51101.3us
W20260709 12:13:57.381214 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.385633 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:57.386420 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.388284 23925 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:57.333101 26259 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.389393 23930 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:57.389549 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.412009 23855 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.421886 26146 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.422457 26146 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.423069 23930 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:57.381242 26226 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:57.426855 23925 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:57.434353 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.394490 25843 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:57.428604 26099 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:13:57.439055 26288 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:13:57.444932 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.445832 25169 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:57.447854 26092 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:13:57.448809 26279 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 12:13:57.440297 26099 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.448964 26092 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:57.451490 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.452052 26157 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:57.426254 26259 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:13:57.453455 26246 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:57.456282 26296 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.427718 26226 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.459201 26146 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.462913 25169 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:13:57.465777 25925 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.428184 26087 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.471958 23926 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:57.436365 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.489516 23860 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:13:57.490717 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 164342us
I20260709 12:13:57.509512 23927 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:57.487480 26092 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:57.488881 25843 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:57.504989 25169 pending_rounds.cc:85] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 5
I20260709 12:13:57.506467 26296 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.515681 23930 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.505498 26226 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.516485 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:57.506201 25987 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:57.522050 26297 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:57.522549 26297 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.522888 26297 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:57.533371 26296 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.545936 23855 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:57.554659 23926 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:13:57.559305 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.557224 26046 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:13:57.550971 26225 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:57.562026 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.562780 26225 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.551030 26036 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:57.569120 26036 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:57.570117 26036 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.517841 25169 pending_rounds.cc:107] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:13:57.580654 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.583110 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.584726 23927 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.557984 26296 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:57.576043 26146 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:57.574901 25169 raft_consensus.cc:2889] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:57.571552 26036 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:57.613276 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 96881.6us
I20260709 12:13:57.597316 26146 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:57.576510 25843 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.601938 26259 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:57.634649 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.575274 26092 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.672401 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:57.685619 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 49458.8us
I20260709 12:13:57.638777 26239 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.611853 26276 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.696465 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.674646 26296 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:57.702138 23778 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
W20260709 12:13:57.708364 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 241130us
I20260709 12:13:57.701790 26225 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.717583 23934 raft_consensus.cc:2368] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 6.
I20260709 12:13:57.693531 26259 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:57.722401 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 297230us
W20260709 12:13:57.724022 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 235435us
I20260709 12:13:57.699687 26302 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.731274 23855 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.732812 23853 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:57.734273 25978 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:57.734632 26266 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:57.735176 25978 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:57.735615 26266 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:57.735530 25978 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:57.736130 26266 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:57.736709 25978 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.736995 26266 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.737931 25978 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.737905 26266 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.728947 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 271) took 4964 ms. Trace:
I20260709 12:13:57.720350 26092 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:57.740408 23683 rpcz_store.cc:276] 0709 12:13:52.764147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.764323 (+   176us) service_pool.cc:224] Handling call
0709 12:13:57.728889 (+4964566us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:56.756438 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:56.887498 (+131060us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:56.887554 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:13:56.887575 (+    21us) tablet.cc:662] Decoding operations
0709 12:13:56.888097 (+   522us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:56.888169 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:56.888197 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:56.888662 (+   465us) tablet.cc:701] Row locks acquired
0709 12:13:56.888710 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:13:56.888903 (+   193us) write_op.cc:270] Start()
0709 12:13:56.889073 (+   170us) write_op.cc:276] Timestamp: P: 1783599236888828 usec, L: 0
0709 12:13:56.889096 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:13:56.889878 (+   782us) log.cc:844] Serialized 918 byte log entry
0709 12:13:57.062495 (+172617us) op_driver.cc:464] REPLICATION: finished
0709 12:13:57.274362 (+211867us) write_op.cc:301] APPLY: starting
0709 12:13:57.274522 (+   160us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:57.274709 (+   187us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:57.274888 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:57.275401 (+   513us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:57.388074 (+112673us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:13:57.395443 (+  7369us) write_op.cc:312] APPLY: finished
0709 12:13:57.442868 (+ 47425us) log.cc:844] Serialized 29 byte log entry
0709 12:13:57.443039 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:57.443130 (+    91us) write_op.cc:454] Released schema lock
0709 12:13:57.443170 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:57.443611 (+   441us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:57.443637 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":55787,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":166924,"apply.run_cpu_time_us":2832,"apply.run_wall_time_us":176426,"num_ops":2,"prepare.queue_time_us":124063,"prepare.run_cpu_time_us":3693,"prepare.run_wall_time_us":140695,"replication_time_us":153588,"thread_start_us":167369,"threads_started":2}]]}
I20260709 12:13:57.728340 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.693900 26303 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:57.695827 26239 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.747084 26259 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:57.754884 23929 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.716801 26157 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:13:57.777179 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:57.740146 26087 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.778908 26087 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:57.780360 26299 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:13:57.727087 25925 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.778369 23852 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:57.781250 26087 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.698390 26146 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.725409 26300 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.753381 26174 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:57.754113 26080 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:57.759464 26296 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:57.803124 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.722141 26225 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:13:57.809798 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.812335 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 115912us
I20260709 12:13:57.778275 23682 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.799154 25925 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:13:57.814792 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 99891.6us
I20260709 12:13:57.817797 23682 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.819548 23682 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 3 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.821579 23682 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.824126 23682 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 3 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.825059 26146 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.825490 26239 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.826152 23682 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 1 to 2, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 2 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:57.826759 26259 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.827764 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:57.829190 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 100434us
I20260709 12:13:57.837380 26276 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.829071 26225 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.848344 26204 raft_consensus.cc:1064] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:57.848532 26080 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.851048 26080 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.827777 26306 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:57.855415 26306 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:57.856630 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:57.753850 26302 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.857533 26225 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:57.858404 26225 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.859061 26302 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:57.856369 26306 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.858944 26225 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.860204 26306 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.815729 26300 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:57.861474 26225 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.812341 26307 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:57.863058 26307 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:57.864039 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.863860 23855 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
W20260709 12:13:57.864017 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 105030us
I20260709 12:13:57.869263 23855 raft_consensus.cc:606] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:57.831280 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:57.871752 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:13:57.824085 26301 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:13:57.880612 23749 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:57.881860 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:57.862533 26300 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.874163 26225 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.893534 23824 leader_election.cc:400] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:57.874064 26046 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:57.896412 23824 leader_election.cc:403] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:57.879943 25987 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:57.876327 25925 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.894583 26296 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:57.916903 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:57.917213 26174 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:57.917601 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:57.924517 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:57.896783 26302 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.904083 25987 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.934187 25987 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.935545 26094 raft_consensus.cc:1064] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:57.903081 26157 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.937016 26157 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.938022 26094 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.931334 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:57.940291 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.940654 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:57.931236 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45377.9us
I20260709 12:13:57.929490 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:57.943181 25174 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:57.939358 26157 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.944002 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:57.900767 26225 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.947806 25176 raft_consensus.cc:2368] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:13:57.943674 23777 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:57.942636 26302 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.949278 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:57.950541 25173 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.950649 26225 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.951068 26302 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.915524 26161 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:13:57.948280 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:57.962203 23776 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.963354 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.964197 23782 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:57.964542 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.965257 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.965673 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.966302 23776 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:57.967478 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:57.968240 23784 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:57.970034 23928 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:57.974983 25173 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:57.965883 23782 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:57.976126 25987 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:57.978461 26297 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:57.979153 25987 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.979521 23778 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:57.980507 26307 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:57.964416 23780 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:57.991461 26204 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:57.995133 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 102026us
I20260709 12:13:57.989701 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:57.998140 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52995us
I20260709 12:13:57.999402 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:58.008170 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:58.008744 23860 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:58.010373 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.007579 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:58.025084 23785 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:58.040432 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.041162 23777 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.005204 26117 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.004927 26161 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.006544 25843 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:58.059996 23778 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:13:58.050609 26161 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.058918 26313 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:58.079802 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 65437us
I20260709 12:13:58.083799 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.051380 26117 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:13:58.088025 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.067000 26161 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.079370 26313 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.068536 25987 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.096303 23853 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:58.065834 26052 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:13:58.100669 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:58.103739 23860 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:13:58.105173 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:58.062105 25843 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.113030 23928 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:58.116083 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:58.116479 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.117182 23778 raft_consensus.cc:2368] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:13:58.117688 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:58.122645 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:58.095737 26313 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.137053 23778 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:58.120132 25967 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:58.144572 25967 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:58.145519 25967 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:58.143687 26318 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:13:58.150152 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 100024us
W20260709 12:13:58.163371 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.172046 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.083669 26314 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:13:58.107734 25987 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.125653 26297 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.178269 23778 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.181475 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.178539 26331 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.148811 26313 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.167719 26326 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:58.201951 26315 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:13:58.202123 26117 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.203962 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.136039 26157 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:13:58.198591 23777 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:58.212112 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.198557 26313 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.218143 23776 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.213182 26326 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:13:58.227236 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61650.2us
I20260709 12:13:58.218501 26313 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.225072 26326 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:58.234318 26316 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:13:58.238956 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.235668 25987 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.243290 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.228000 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:58.241119 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.249377 23776 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:13:58.250559 25114 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:58.258503 23853 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:13:58.260169 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 111108us
I20260709 12:13:58.222940 26094 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:13:58.263429 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.265357 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.264240 23776 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:13:58.228223 26320 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:13:58.271556 26332 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:58.272045 26332 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.272883 26332 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:13:58.283471 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 159326us
I20260709 12:13:58.285285 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.263370 25967 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:13:58.292243 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.293004 23776 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.274091 26332 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:58.279290 26309 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:58.308204 26309 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:58.310851 26309 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.291270 25967 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.311930 25967 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:58.274904 26314 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:58.321372 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:58.321915 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:58.322331 23930 raft_consensus.cc:2410] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:13:58.324575 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.321540 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:58.262444 26161 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.288130 26281 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:13:58.392426 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:58.408624 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.410542 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 82515.5us
W20260709 12:13:58.411823 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.401237 26314 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.417613 26314 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.418294 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:58.419077 25168 raft_consensus.cc:2368] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:58.419973 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:58.408397 26266 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:58.421082 26266 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.427166 26266 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.416075 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:58.429819 25967 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.433694 26326 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.436057 26326 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.424494 26336 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.438202 26336 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.441569 26336 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.442615 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:58.443446 26331 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:58.448989 25168 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:58.449627 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.450387 23780 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:13:58.450809 23898 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:58.452212 26266 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:58.452629 26266 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:58.452977 26266 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:58.456008 23898 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:58.456776 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:58.404754 26226 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.016s
W20260709 12:13:58.471709 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56438.6us
I20260709 12:13:58.471997 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:58.473623 26080 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:58.474222 26080 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:58.475170 26080 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.436118 25170 raft_consensus.cc:2368] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
W20260709 12:13:58.471683 26316 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:13:58.462092 26161 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.483546 26313 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.485436 26338 raft_consensus.cc:993] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:13:58.485999 26338 raft_consensus.cc:1081] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:13:58.478657 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 214477us
I20260709 12:13:58.451537 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:58.488708 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:58.490592 23926 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.483425 25843 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.495952 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:13:58.477380 26281 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:58.507825 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.508574 23780 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:13:58.477037 26226 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.511343 25114 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:58.516355 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:13:58.520386 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.489310 26313 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:58.522212 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.512413 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:58.512790 26092 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:58.514405 25843 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:13:58.517602 26309 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:58.497040 26094 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:13:58.532338 25173 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:13:58.535801 26309 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.538059 26309 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.543071 25174 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:58.527530 23926 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:58.545207 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:58.514550 25169 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:58.554718 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:58.541286 26326 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:58.557090 26320 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:58.557891 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:58.582768 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 178932us
I20260709 12:13:58.583354 23783 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.559396 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.561161 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.544258 26092 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.570130 25168 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:13:58.569813 26266 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.553005 26161 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:58.572196 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.592995 25173 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:58.526475 25967 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.598841 25174 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:58.601789 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.602476 23777 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:58.604015 23780 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:13:58.604406 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.605082 23777 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:58.606400 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.582547 26336 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.613524 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.551178 26094 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:13:58.585301 26332 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:58.581543 26326 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.616560 23777 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:58.611294 25114 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:58.614907 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.618557 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.625538 23780 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:13:58.632750 23782 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:58.640090 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.640358 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.640914 23854 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:13:58.641337 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.641754 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.642292 23824 leader_election.cc:400] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:13:58.642740 23824 leader_election.cc:403] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:13:58.642999 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.643360 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.643936 23784 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:58.644037 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.645174 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.646143 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:58.646945 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:58.642385 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:58.649966 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.612233 26161 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:58.652150 26161 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.636907 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.656378 23785 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:58.642783 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:13:58.661109 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51831.7us
I20260709 12:13:58.667330 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:58.668095 23860 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:13:58.675172 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.612030 26329 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:13:58.657140 25967 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:13:58.661053 26338 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:13:58.684545 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 186649us
I20260709 12:13:58.651225 26335 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 12:13:58.700665 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 134437us
I20260709 12:13:58.694341 23782 pending_rounds.cc:85] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:13:58.710139 23782 pending_rounds.cc:107] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:13:58.710438 23782 raft_consensus.cc:2889] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:58.714630 23784 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:13:58.721828 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.684582 25170 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:58.722610 23776 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.727962 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.728760 23784 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.733155 26314 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:58.733772 26314 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.734750 26314 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.736723 26314 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.735791 23777 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:58.740465 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.742655 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.746433 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.748801 23777 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:58.693029 26309 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:58.768998 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.692847 26281 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:58.691727 25967 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:58.687759 26094 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:13:58.781807 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:58.783522 23784 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:58.773258 26332 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.693763 26338 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:58.818965 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 176801us
I20260709 12:13:58.819965 23777 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:58.789645 26092 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.793524 26337 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:13:58.834432 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.836071 23777 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:58.787994 26226 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:58.831820 23784 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:13:58.693439 26326 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:58.951109 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.955201 23784 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.953557 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:58.957103 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 188071us
I20260709 12:13:58.957880 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:58.785092 26281 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:58.795027 26174 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260709 12:13:58.794732 26309 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:13:58.960757 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.979382 23784 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.971192 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.971941 23749 leader_election.cc:400] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:58.980146 23777 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:58.980181 23749 leader_election.cc:403] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:58.984666 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:58.985733 23749 leader_election.cc:400] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:13:58.989960 26337 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:58.990458 26337 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:58.992203 26336 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.992702 26336 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.996025 26297 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:58.967545 26226 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.967794 26266 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.000258 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:58.960283 26052 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:59.012780 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 178289us
I20260709 12:13:59.013674 23783 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:13:58.986042 23749 leader_election.cc:403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:13:59.008819 23784 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:59.011137 23776 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:13:59.029835 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.030026 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.030630 23784 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:13:59.029331 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:59.032727 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.036062 23785 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:13:59.041280 23855 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:59.043900 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.037964 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:59.055472 23785 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:59.057466 23784 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:58.992090 26174 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:13:59.061477 23855 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:59.016386 26296 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4.
I20260709 12:13:59.063905 26344 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:58.991272 26337 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.065073 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.065825 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:59.067162 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.072724 26307 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:13:59.072952 23776 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:59.076947 26092 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:13:59.090588 26092 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:59.073985 26359 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 12:13:59.092835 26092 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.093514 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2162) took 1132 ms. Trace:
I20260709 12:13:59.082719 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:13:59.094936 23855 raft_consensus.cc:2376] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7. Re-sending same reply.
W20260709 12:13:59.083083 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50135.3us
I20260709 12:13:59.093926 23781 rpcz_store.cc:276] 0709 12:13:57.961042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.966127 (+  5085us) service_pool.cc:224] Handling call
0709 12:13:57.966456 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:57.985392 (+ 18936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.985423 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.986510 (+  1087us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.240718 (+254208us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b18003e3b60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5522 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:13:58.293666 (+ 52948us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b18003e3b60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9
0709 12:13:58.506075 (+212409us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18003e3b60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:13:58.672591 (+166516us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:58.672626 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.672692 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.093311 (+420619us) raft_consensus.cc:1685] finished
0709 12:13:59.093329 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.093461 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14937,"raft.queue_time_us":18910,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7190,"spinlock_wait_cycles":19055488,"thread_start_us":166313,"threads_started":1,"wal-append.queue_time_us":1047187}
I20260709 12:13:59.096207 26369 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:59.096657 26351 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.097298 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.097162 26351 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.098448 26369 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:59.098896 26369 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:59.099095 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.099843 26370 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:59.100221 26370 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:59.085117 23784 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:59.059850 26326 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:59.069165 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.088511 23783 raft_consensus.cc:2376] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:13:59.051148 26340 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.087945 23860 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5. Re-sending same reply.
I20260709 12:13:58.974783 25967 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.101043 26370 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.068805 26306 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:59.059046 26052 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:59.017491 26226 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.123740 23777 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:59.119815 26369 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.093441 26366 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.004231 26332 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.082399 26344 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:13:59.104920 26348 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
I20260709 12:13:59.126307 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:59.132011 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.104359 26352 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.118603 26117 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:59.119410 26307 raft_consensus.cc:2764] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:13:59.125622 26092 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:59.161247 26092 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.161669 26092 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:59.165524 26352 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.167949 26352 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.168540 26352 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:59.154842 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.159716 26373 raft_consensus.cc:1064] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:13:59.169486 26052 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.170954 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.167399 26351 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.171980 26351 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:59.172269 26373 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.169544 26370 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:59.174413 26351 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:59.175309 26351 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:13:59.153098 26117 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.181305 26117 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.160737 26374 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.182736 26374 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.184885 26226 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.145854 26372 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:59.149158 26306 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:13:59.187656 26306 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.172207 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.190732 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.193786 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:13:59.195747 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 210140us
I20260709 12:13:59.197465 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:13:59.202036 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 214291us
I20260709 12:13:59.159826 26337 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.216770 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:13:59.138540 26340 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.217177 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.195124 26352 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:59.184923 26374 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.201635 26372 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:59.214479 26329 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.004s	sys 0.002s
I20260709 12:13:59.175901 26080 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:59.215616 26259 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:59.214375 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.251029 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:59.297763 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 172) took 1298 ms. Trace:
I20260709 12:13:59.424211 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:59.382439 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 344483us
W20260709 12:13:59.436923 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 208369us
W20260709 12:13:59.335069 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.435169 23927 rpcz_store.cc:276] 0709 12:13:57.999329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.000242 (+   913us) service_pool.cc:224] Handling call
0709 12:13:58.012058 (+ 11816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.023534 (+ 11476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.023566 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.343100 (+319534us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b2c000bb9c8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:13:58.412622 (+ 69522us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.586385 (+173763us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:58.610143 (+ 23758us) log.cc:844] Serialized 14 byte log entry
0709 12:13:58.610376 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.610459 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:58.676459 (+ 66000us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b18003f3100. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:13:59.062603 (+386144us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b18003f3100. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5522 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:13:59.297512 (+234909us) raft_consensus.cc:1685] finished
0709 12:13:59.297537 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.297697 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7427,"prepare.queue_time_us":23381,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":3642,"raft.queue_time_us":25919,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":20548,"spinlock_wait_cycles":27071360,"thread_start_us":106471,"threads_started":1,"wal-append.queue_time_us":213636}
I20260709 12:13:59.438822 26259 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:59.439793 26259 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:59.440927 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 190015us
I20260709 12:13:59.436738 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.297341 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.439865 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.442608 23930 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:13:59.220685 26266 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:59.447691 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.448360 23860 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:13:59.449746 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.257256 26365 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:59.221114 25843 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.252295 23934 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.217829 26297 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:59.225514 26344 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.288764 26378 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:59.338508 26363 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:13:59.324105 26368 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:13:59.221261 26204 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:13:59.313392 26338 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.307535 26372 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:13:59.333343 26336 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:59.331629 26356 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
W20260709 12:13:59.337283 26349 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
W20260709 12:13:59.320226 26358 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.245s	user 0.004s	sys 0.004s
I20260709 12:13:59.403988 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.291828 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.521476 23926 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:59.150616 26366 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.288882 26364 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:13:59.315332 26340 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.442876 23933 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:59.335047 26362 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:59.446374 23929 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:13:59.447052 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.459084 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:13:59.463430 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.471951 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.472807 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.507428 25967 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:59.502118 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:13:59.530229 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66901us
I20260709 12:13:59.440858 26329 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:59.536751 25169 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:59.543440 26092 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.508453 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.509155 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.545578 23855 raft_consensus.cc:2368] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6.
I20260709 12:13:59.509523 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.511912 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.509127 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.515239 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.546635 23777 raft_consensus.cc:2376] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
I20260709 12:13:59.546882 23860 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:59.516409 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.447507 26266 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:59.516119 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.002s
W20260709 12:13:59.549347 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.006s	sys 0.002s
I20260709 12:13:59.522269 23928 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:59.503551 26080 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:59.552742 23783 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:13:59.526510 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.527228 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.528555 23785 raft_consensus.cc:2368] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:13:59.499099 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:59.499644 26373 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 12:13:59.528842 26371 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:13:59.529531 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70504.3us
I20260709 12:13:59.506364 26307 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:59.556162 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 57168.3us
I20260709 12:13:59.531210 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.531798 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:13:59.533942 26309 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.534096 26384 raft_consensus.cc:2764] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:13:59.535719 26333 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:59.562373 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:13:59.548079 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.547257 26344 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.517941 26378 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:59.553653 23926 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.567911 23928 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:13:59.552574 23779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.017s	sys 0.054s
I20260709 12:13:59.558341 23784 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:59.590785 25843 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:59.560240 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.591248 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.548563 23925 raft_consensus.cc:2410] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:13:59.561131 23931 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:13:59.592185 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:13:59.592833 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.548776 26351 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:59.593375 23931 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:59.591533 23933 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:13:59.593858 26326 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:59.594262 26326 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:59.565363 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.593405 26204 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:59.595065 23854 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:13:59.569869 26382 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.571800 26389 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:13:59.572583 26372 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.573130 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.573913 26378 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.575100 26384 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:59.567070 26379 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:13:59.564745 26374 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:13:59.562333 26352 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:59.577543 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:59.573791 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:13:59.580794 23928 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:59.571874 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.581354 26336 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.582559 26369 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.583402 26307 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.585075 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 217) took 1078 ms. Trace:
I20260709 12:13:59.624631 23778 rpcz_store.cc:276] 0709 12:13:58.506949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.509319 (+  2370us) service_pool.cc:224] Handling call
0709 12:13:58.509657 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.517777 (+  8120us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:58.548628 (+ 30851us) log.cc:844] Serialized 14 byte log entry
0709 12:13:58.786822 (+238194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.794876 (+  8054us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.795020 (+   144us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:58.795052 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.795118 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.319659 (+524541us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:13:59.584740 (+265081us) raft_consensus.cc:1685] finished
0709 12:13:59.584856 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.585012 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33456,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":4666,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":17653,"spinlock_wait_cycles":22470912,"thread_start_us":172068,"threads_started":1,"wal-append.queue_time_us":471562}
I20260709 12:13:59.626308 23930 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:59.589998 23934 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:59.566926 26333 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.552892 26266 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.594615 26326 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:59.590432 23779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.017s	sys 0.054s
I20260709 12:13:59.562518 26329 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:59.598331 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.636970 23777 raft_consensus.cc:2376] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6. Re-sending same reply.
I20260709 12:13:59.547526 25967 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.620150 23855 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:59.620261 23931 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:13:59.637226 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:13:59.620676 23782 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:13:59.627579 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70609.9us
W20260709 12:13:59.631516 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54054.5us
I20260709 12:13:59.638736 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:59.633961 23784 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:13:59.616109 25843 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.635325 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:13:59.644090 23780 raft_consensus.cc:2376] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:13:59.646090 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:13:59.636250 23926 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:13:59.637931 23778 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:13:59.641342 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:13:59.642398 23857 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:59.618443 26389 consensus_queue.cc:1243] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:13:59.661048 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:59.661475 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 101700us
I20260709 12:13:59.647213 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.662596 23777 raft_consensus.cc:2368] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:13:59.608071 26331 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:13:59.664386 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.633337 26351 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:59.681797 23926 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:59.674201 23934 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:59.683136 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.684021 23926 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:13:59.730967 23926 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:13:59.613098 26297 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:13:59.739259 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 100149us
I20260709 12:13:59.677048 26174 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:59.675114 26351 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:13:59.740888 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2218) took 1065 ms. Trace:
I20260709 12:13:59.748104 23779 rpcz_store.cc:276] 0709 12:13:58.675075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.675954 (+   879us) service_pool.cc:224] Handling call
0709 12:13:58.676300 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.676574 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.676600 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.683421 (+  6821us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.025556 (+342135us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:59.643971 (+618415us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.740523 (+ 96552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.740605 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.740659 (+    54us) raft_consensus.cc:1685] finished
0709 12:13:59.740671 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.740820 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31110,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"thread_start_us":980819,"threads_started":3,"wal-append.queue_time_us":451712}
I20260709 12:13:59.660204 25843 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.752399 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:13:59.737494 26368 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:13:59.752113 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.750140 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:13:59.745939 26359 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:13:59.738977 26364 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:13:59.753899 26351 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:59.781046 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.761338 26174 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:59.743057 26092 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.773551 26361 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.142s
I20260709 12:13:59.611354 26382 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.783897 23779 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:59.738692 26204 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:13:59.794590 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 91294.9us
I20260709 12:13:59.798043 23779 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:59.785281 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:13:59.801682 23777 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:59.802070 23777 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:59.804873 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.816639 23779 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:59.825577 26366 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:13:59.826691 26392 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:13:59.828114 26394 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:59.828722 26394 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:59.829641 26394 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.831930 26377 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 12:13:59.834709 26281 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.803967 26365 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.838645 26365 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.840935 26365 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.842574 23933 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:59.844779 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:13:59.838968 26320 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:59.847095 26320 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:13:59.848707 26382 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:13:59.849495 26320 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.850927 26320 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:59.852135 26306 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.847026 25174 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:59.795296 26384 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:13:59.860663 26396 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:13:59.790879 26338 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:13:59.861544 26381 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.862856 25843 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:13:59.864890 26307 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.868227 26333 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.874395 25967 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.797181 26399 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:59.875433 26369 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:13:59.878245 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.879413 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.807391 26362 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 12:13:59.939427 23929 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:13:59.800622 26364 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.954022 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.931173 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:13:59.944413 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 202403us
I20260709 12:13:59.946714 23933 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 2.
W20260709 12:13:59.955575 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 170364us
I20260709 12:13:59.956280 23900 leader_election.cc:400] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6.
I20260709 12:13:59.956698 23900 leader_election.cc:403] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:59.941726 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 279307us
I20260709 12:13:59.929170 26080 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:13:59.953505 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.954115 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:13:59.954710 23929 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:13:59.952942 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 311648us
W20260709 12:13:59.947469 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 102668us
I20260709 12:13:59.852618 26306 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.969333 23926 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:59.987394 23931 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:13:59.952659 26365 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:13:59.989095 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:13:59.984257 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:13:59.952447 26399 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:13:59.958204 26369 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.997485 23900 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:00.028146 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 43990.8us
W20260709 12:14:00.033272 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.941623 26381 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:13:59.929898 26384 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:13:59.989166 26365 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:00.047957 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 241942us
I20260709 12:13:59.947340 26174 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.032837 26377 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:00.032469 26333 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:00.066511 26407 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:00.072144 26410 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:00.072613 26410 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.072956 26410 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:00.075706 26393 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6.
I20260709 12:14:00.076004 26365 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.077261 26365 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:00.077373 26399 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:00.088166 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.095300 26333 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.140570 26364 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:00.035064 26369 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.071065 26266 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:00.268204 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 300160us
I20260709 12:14:00.291797 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.303543 23853 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:00.291303 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:00.292076 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.304395 23855 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:00.271032 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.275812 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 248211us
I20260709 12:14:00.304665 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:00.269021 23934 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:00.057623 26377 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:00.265210 26344 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.669s	user 0.004s	sys 0.005s
I20260709 12:14:00.268637 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:00.310813 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:00.308454 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:00.266151 26400 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.320406 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:00.323004 23932 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:14:00.325206 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:00.325732 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 57214.6us
I20260709 12:14:00.328760 25174 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:00.288260 26320 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.142647 26384 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.328434 23856 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:00.287890 26364 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.279150 26379 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:00.214111 26365 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.326175 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:00.329103 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:00.282789 23777 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.339306 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.339771 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:00.340425 23825 leader_election.cc:400] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:00.340739 23825 leader_election.cc:403] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:00.259418 26405 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:00.337363 26296 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:00.301352 26333 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:00.327152 23749 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.668s (SENT) [suppressed 22 similar messages]
I20260709 12:14:00.342314 23899 leader_election.cc:400] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:14:00.343127 23899 leader_election.cc:403] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:00.344041 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:00.335909 26326 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.003s
I20260709 12:14:00.323935 26381 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.332576 26372 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:00.352056 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.347535 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.350286 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:00.334276 26393 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:00.341574 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:00.360142 26384 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:00.345026 25843 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.361116 25843 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.361503 25843 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.362452 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:00.347115 26296 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.341533 26329 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:00.364619 26329 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.365535 26376 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:00.365963 26376 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:00.366786 26329 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.368333 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:00.368690 26266 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:00.368973 23779 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:00.369853 23931 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:00.347251 26406 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.352138 26382 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.371958 26382 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:00.372359 26382 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.356153 26306 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.342036 26364 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.360737 26384 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.345055 26344 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.379189 23855 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:00.380440 23856 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:00.345232 26407 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:00.402483 23857 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:14:00.356856 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:00.403416 25174 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:14:00.397478 23781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.006s	sys 0.017s
I20260709 12:14:00.366910 26376 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:00.403985 23781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.006s	sys 0.018s
I20260709 12:14:00.365415 26296 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.335216 26320 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.404826 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.417091 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:00.355079 26372 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.418509 23852 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:00.418867 23852 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:00.397924 26351 consensus_meta.cc:287] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 12:14:00.342828 26379 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.334570 26400 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.346426 26365 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.421885 23858 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:00.395471 26398 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.354948 26326 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.425084 23852 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.454123 23852 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.398814 26266 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:00.395885 26338 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:00.409047 26407 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:00.398491 26393 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:14:00.416934 26402 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:00.421341 26372 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:00.425109 26376 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:00.405175 26414 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:00.425297 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:00.414482 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:00.464103 23854 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:00.463529 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49138.3us
I20260709 12:14:00.466318 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:00.443212 26423 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.419206 26362 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:14:00.442720 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:00.471161 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 125470us
I20260709 12:14:00.413277 26344 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.509496 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:00.523694 26399 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.525777 26424 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:00.349291 26405 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.458348 26338 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.526355 26424 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:00.535753 25174 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:00.430464 26351 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.458227 26407 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.454114 26398 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.507620 25168 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:00.565786 26365 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:00.527828 26378 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:00.525547 26266 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:00.576108 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 225868us
I20260709 12:14:00.584633 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:00.555794 26338 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:00.586802 23928 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:00.400413 26377 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.552026 26420 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:00.595954 23852 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:00.564566 26369 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.551641 26329 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:14:00.603487 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:00.606240 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.536703 26296 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:14:00.615865 23783 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.566568 26398 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:00.621032 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.507608 26423 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.627669 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:00.628559 26422 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 12:14:00.630165 26421 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:14:00.630447 26389 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:00.630923 26389 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:00.634227 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:00.635035 23785 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:00.636384 23785 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:00.639634 23780 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:00.640048 23780 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:00.640797 23780 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.641760 23780 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.464627 26402 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:00.627434 26417 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.199s
W20260709 12:14:00.701109 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.011296 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:00.746451 23779 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:14:00.649019 26419 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:14:00.688889 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 217086us
I20260709 12:14:01.012843 26373 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.753432 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.603662 26420 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.042953 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:00.614463 26094 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:00.726758 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.023000 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.868606 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2167) took 1314 ms. Trace:
I20260709 12:14:00.745643 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.015934 23777 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.602141 26425 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.011914 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.010669 26340 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.758694 25169 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:14:01.024662 26306 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.032799 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.066414 23857 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:14:01.009347 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 543064us
I20260709 12:14:01.021275 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 240) took 1055 ms. Trace:
I20260709 12:14:01.018467 26378 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.034613 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 450021us
I20260709 12:14:01.069801 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:00.759382 26326 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.035849 26436 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.036170 26394 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.037536 26080 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.037475 26393 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.869438 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2177) took 1247 ms. Trace:
I20260709 12:14:00.804751 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2162) took 1254 ms. Trace:
I20260709 12:14:01.048424 26420 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:00.745584 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:00.706488 26400 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.021346 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.079576 23934 rpcz_store.cc:276] 0709 12:13:59.622236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.703710 (+ 81474us) service_pool.cc:224] Handling call
0709 12:13:59.743092 (+ 39382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.309479 (+566387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.869152 (+559673us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b540006b3d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:00.869363 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209014528}
I20260709 12:14:01.041399 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.041471 26338 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.084955 23925 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:01.009913 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.043880 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.069605 26384 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.068792 26437 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.097438 26437 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.076026 26430 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:01.100234 26430 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:01.101260 26430 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:01.049120 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.104467 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.104683 26372 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.005s
W20260709 12:14:00.881186 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2249) took 1403 ms (client timeout 1864 ms). Trace:
I20260709 12:14:01.050793 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.105279 26372 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.107009 26383 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.050912 23859 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:14:01.108911 23784 rpcz_store.cc:269] 0709 12:13:59.478173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.590984 (+112811us) service_pool.cc:224] Handling call
0709 12:14:00.880958 (+1289974us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b540011bcd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:00.881087 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96696576}
I20260709 12:14:01.107708 26383 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.112443 26383 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.076417 26378 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.053535 26373 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.049072 23932 rpcz_store.cc:276] 0709 12:13:59.554452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.564286 (+  9834us) service_pool.cc:224] Handling call
0709 12:13:59.564621 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:00.328462 (+763841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.868224 (+539762us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54002445d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:00.868517 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743547264}
I20260709 12:14:01.052090 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:00.867712 26432 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:14:00.929580 23783 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:01.055080 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:00.878913 26410 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.806s	user 0.000s	sys 0.008s
W20260709 12:14:00.826952 26386 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 1.256s	user 0.000s	sys 0.005s
I20260709 12:14:01.058494 26340 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.619228 26320 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:00.643489 26052 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.059007 23778 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:01.133265 23778 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:14:00.752355 26377 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.061244 23780 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:14:01.135867 23780 pending_rounds.cc:85] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 6
I20260709 12:14:01.136129 23780 pending_rounds.cc:107] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:14:01.136368 23780 raft_consensus.cc:2889] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.137034 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 238) took 1351 ms. Trace:
W20260709 12:14:01.048179 25843 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.001s
I20260709 12:14:01.067140 23779 rpcz_store.cc:276] 0709 12:13:59.966134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.716760 (+750626us) service_pool.cc:224] Handling call
0709 12:14:00.717126 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.018325 (+301199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.021206 (+  2881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:01.121145 26384 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.712771 26351 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:00.710003 26406 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4.
I20260709 12:14:00.624040 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.139380 26052 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.139876 25168 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:00.755864 26364 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:00.720520 26344 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.069963 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43422 (request call id 2190) took 1420 ms. Trace:
I20260709 12:14:01.071069 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:01.141610 25843 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.035225 26398 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.649267 26296 consensus_queue.cc:1243] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:01.142416 23855 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
W20260709 12:14:01.035442 23776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2243) took 1560 ms (client timeout 1699 ms). Trace:
W20260709 12:14:01.053926 26382 consensus_meta.cc:287] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.001s
I20260709 12:14:00.802883 26405 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:00.706410 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 845) took 1088 ms. Trace:
I20260709 12:14:01.048381 26297 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.143342 26382 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.143666 26297 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.144807 25843 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.145323 26382 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.145424 25843 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:01.145910 26297 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.146164 23933 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:01.048293 26094 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:01.079677 23927 rpcz_store.cc:276] 0709 12:13:59.550205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.550331 (+   126us) service_pool.cc:224] Handling call
0709 12:13:59.559119 (+  8788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:59.627537 (+ 68418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.627560 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:59.627633 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:59.627666 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.804474 (+1176808us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54002436d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:00.804528 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.804673 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134439808}
I20260709 12:14:01.145833 26398 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:01.146931 26434 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:01.079118 26394 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:00.720728 26424 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.089805 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:01.090754 26402 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.125211 26372 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.091044 26400 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:01.084187 26393 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.091719 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.184365 23925 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:01.076464 26306 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.092255 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:01.084055 26080 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:01.095855 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.188139 23931 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:01.074606 26435 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:00.777480 23785 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:14:01.049211 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.119540 23929 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:14:01.121899 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.121816 23860 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:14:01.191819 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.192606 23859 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
W20260709 12:14:01.192698 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 143353us
I20260709 12:14:01.193580 23854 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:01.194624 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.195178 23856 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:00.883939 26266 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.059885 26425 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.116492 26337 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.137384 26259 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:14:01.138636 26384 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.142148 26052 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:00.704826 26369 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.189841 26344 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:01.147550 26398 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.148240 23927 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:01.083443 23852 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:01.137452 23778 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:01.137337 23780 rpcz_store.cc:276] 0709 12:13:59.785414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.639239 (+853825us) service_pool.cc:224] Handling call
0709 12:14:00.639495 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.136801 (+497306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.136985 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:01.206895 26376 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.742s	user 0.006s	sys 0.004s
W20260709 12:14:01.143098 23776 rpcz_store.cc:269] 0709 12:13:59.475371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.475772 (+   401us) service_pool.cc:224] Handling call
0709 12:14:01.035229 (+1559457us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540010c554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:01.035379 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962887552}
I20260709 12:14:01.141376 23777 rpcz_store.cc:276] 0709 12:13:59.633860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.778718 (+144858us) service_pool.cc:224] Handling call
0709 12:14:00.708508 (+929790us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b4c001d7d80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aba7ea20 00007fb2aba8fa29 00007fb2aba9013f 00007fb2aba542ab 00007fb2aba81f3c 00007fb2aba823fe 00007fb2abd0f88b 00007fb2ad70e495 00007fb2ad75f30f 00007fb2ad7a9a55 00007fb2b2779cd0
0709 12:14:01.049675 (+341167us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":195,"spinlock_wait_cycles":58567552}
I20260709 12:14:01.148893 23928 rpcz_store.cc:276] 0709 12:13:59.618120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.633754 (+ 15634us) service_pool.cc:224] Handling call
0709 12:14:00.706190 (+1072436us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b5400202bd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:00.706318 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2125239680}
I20260709 12:14:01.150015 26437 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.145855 25843 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:01.158125 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2197) took 1522 ms. Trace:
I20260709 12:14:01.077907 26436 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.128341 26410 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:01.186677 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 94505.9us
I20260709 12:14:01.190205 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 236) took 1489 ms. Trace:
I20260709 12:14:01.226549 23785 rpcz_store.cc:276] 0709 12:13:59.700932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.637340 (+936408us) service_pool.cc:224] Handling call
0709 12:14:00.637609 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.189922 (+552313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.190146 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:01.160507 26423 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.207865 26320 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.198287 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:01.191896 26407 consensus_meta.cc:287] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.612s	user 0.003s	sys 0.007s
I20260709 12:14:01.079463 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.212592 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.191327 26400 consensus_queue.cc:1243] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:01.190455 26340 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:01.142416 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71417.9us
I20260709 12:14:01.184402 26246 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:14:01.218036 23926 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:01.247551 23928 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:01.222512 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:01.222831 23781 rpcz_store.cc:276] 0709 12:13:59.635749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.879653 (+243904us) service_pool.cc:224] Handling call
0709 12:13:59.880016 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.880458 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.880491 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.880979 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:14:00.409499 (+528520us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:00.409538 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.409602 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:01.157898 (+748296us) raft_consensus.cc:1685] finished
0709 12:14:01.157919 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:01.158063 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7178,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11243,"thread_start_us":528165,"threads_started":1,"wal-append.queue_time_us":590561}
I20260709 12:14:01.208768 26364 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:01.197995 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.016s	sys 0.096s
W20260709 12:14:01.251758 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.017s	sys 0.096s
I20260709 12:14:01.224849 25843 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.208073 26337 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.228757 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:01.210916 26362 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.214592 26344 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:01.236769 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 157391us
I20260709 12:14:01.239573 23751 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:01.232177 23851 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:01.242376 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.249666 23930 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:01.205813 26435 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.223129 26369 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.230433 26376 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.224326 25967 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:14:01.244012 26320 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.414006 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.357193 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.424160 23859 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:01.425439 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.414417 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:01.426549 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 183012us
I20260709 12:14:01.427135 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.427718 23857 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.413991 26094 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.258041 26080 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.256116 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.613516 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:01.328898 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:01.613018 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 373513us
I20260709 12:14:01.607792 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 242) took 1637 ms. Trace:
I20260709 12:14:01.247102 26405 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.611989 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:01.609601 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.438427 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.453176 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 857) took 1396 ms. Trace:
I20260709 12:14:01.611162 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.399845 25171 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:14:01.494292 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.011s	sys 0.004s
W20260709 12:14:01.625844 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.012s	sys 0.012s
I20260709 12:14:01.251401 26297 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:01.536710 23931 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:01.240744 26407 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.455948 25843 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:01.366199 26399 consensus_meta.cc:287] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.842s	user 0.006s	sys 0.002s
W20260709 12:14:01.372475 26427 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.290s	user 0.003s	sys 0.000s
I20260709 12:14:01.449738 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.618913 26362 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:01.685882 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.620862 23933 rpcz_store.cc:276] 0709 12:14:00.056463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.334505 (+278042us) service_pool.cc:224] Handling call
0709 12:14:00.350842 (+ 16337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.146591 (+795749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.452938 (+306347us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b5400203fd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:01.453116 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1828617472}
W20260709 12:14:01.623745 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 397221us
I20260709 12:14:01.620728 23925 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:01.258606 26410 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.256699 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:01.347975 26426 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:14:01.429688 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:01.425370 26400 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.616727 26372 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.533135 26320 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.651976 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:01.453447 26337 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.652597 26425 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.625458 26436 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.340234 26340 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.303s
I20260709 12:14:01.708424 26454 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:14:01.721567 26378 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.669070 23857 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:01.638598 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.722061 26339 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:01.722826 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 84314us
W20260709 12:14:01.631254 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.646246 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.645550 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.724967 26337 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.633278 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.725739 23929 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:01.725462 26337 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.726634 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.630877 26451 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:01.727886 26451 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.730921 26451 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.367339 26338 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.449388 26376 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:01.732214 26453 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:01.733211 26376 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.328063 26382 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.733978 26382 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.734844 26337 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.454035 26393 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.736805 26382 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.737401 26382 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.737613 26398 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.501s	user 0.001s	sys 0.005s
I20260709 12:14:01.737974 26382 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:01.671149 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 60059.1us
I20260709 12:14:01.740032 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:01.741160 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.741979 26456 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:01.742190 26379 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.439046 26373 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.623862 26443 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:01.742779 26378 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.743062 26443 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.743938 26443 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.263528 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.452389 26384 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.613354 26344 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:01.672909 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 448913us
I20260709 12:14:01.752861 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:01.754915 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:01.454180 26394 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:01.637020 23899 leader_election.cc:336] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.699s (SENT)
W20260709 12:14:01.674870 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.676589 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 347729us
I20260709 12:14:01.254657 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.256512 26383 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.415s
I20260709 12:14:01.757757 26383 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:01.758144 26383 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:01.677591 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.677829 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.759915 23853 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:14:01.251163 26423 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.421s
I20260709 12:14:01.759071 26383 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.761821 26383 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.759652 26424 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.409539 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:01.772750 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.773837 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:01.774152 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.778777 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 246) took 1190 ms. Trace:
I20260709 12:14:01.779243 23779 rpcz_store.cc:276] 0709 12:14:00.588562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.138849 (+550287us) service_pool.cc:224] Handling call
0709 12:14:01.139111 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.222913 (+ 83802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:01.222945 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:01.222982 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:01.223010 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.778562 (+555552us) spinlock_profiling.cc:243] Waited 83.6 ms on lock 0x7b5400166554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:01.778602 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:01.778715 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192330112}
W20260709 12:14:01.780424 25171 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 629) took 1445 ms (client timeout 1661 ms). Trace:
I20260709 12:14:01.781143 26435 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.771556 23683 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 3, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 3 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:01.782725 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.783516 23929 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:01.784099 23852 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:01.784741 26414 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.785347 26414 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.695405 23932 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.778851 26377 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.674186 26259 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.686513 23928 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:01.788045 26377 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.401775 26420 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.790264 26377 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.680135 26094 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.681195 26370 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:01.705929 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:01.742374 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.795051 23933 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:01.682812 26434 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.703292 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:01.797978 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.799255 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.799978 23855 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:01.797442 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.435406 23927 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:01.699383 26372 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.687403 26399 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:01.800637 23854 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:01.685113 26405 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:01.662355 26389 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.724512 26340 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:01.257586 26306 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.810729 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.737999 26398 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:01.724555 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.742436 26456 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:01.800366 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 7, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:01.816608 23933 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:01.755520 25843 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.678858 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.707162 26369 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.782980 26407 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:01.795524 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 53219.3us
I20260709 12:14:01.801537 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:01.823935 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:01.824640 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 187652us
I20260709 12:14:01.824651 23927 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:01.824764 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.825964 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.826359 26382 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:01.815112 26410 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.827311 26410 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.819041 26456 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:01.346699 26406 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:01.454481 26445 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:01.264077 26364 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:01.780964 25171 rpcz_store.cc:269] 0709 12:14:00.335381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.362028 (+ 26647us) service_pool.cc:224] Handling call
0709 12:14:01.780161 (+1418133us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54001e8054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:01.780323 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052164480}
I20260709 12:14:01.693259 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.784718 23929 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:01.802201 23930 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:01.839885 23858 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:01.805117 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.806973 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.808492 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.840991 23853 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:01.808568 26373 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.677979 26297 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:01.809612 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 106398us
I20260709 12:14:01.809703 23898 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:01.853070 23682 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 164) took 9452 ms. Trace:
W20260709 12:14:01.811753 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.855324 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.795147 26370 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.799659 26434 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:01.796507 26259 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.819926 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:01.869210 23785 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:01.819574 23683 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:01.820832 23780 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:01.816264 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:01.804672 26372 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.800264 26337 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.823896 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:01.829736 23899 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.864s (SENT)
I20260709 12:14:01.793686 26418 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:14:01.818616 26414 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:01.833271 26406 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.834017 26410 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.846937 26373 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.848321 26389 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.849670 26364 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.855850 25119 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.905891 26434 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.782838 26362 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:01.908200 26344 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.870918 23932 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
W20260709 12:14:01.878742 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 62538.3us
I20260709 12:14:01.878607 25174 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:01.879527 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:01.880553 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 56760.1us
I20260709 12:14:01.877138 23683 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 7, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:01.820724 26405 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.912703 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:01.912794 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.914012 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:01.915750 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59962.1us
I20260709 12:14:01.917114 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:01.912983 23683 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:01.921638 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.922420 23856 raft_consensus.cc:2376] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9. Re-sending same reply.
I20260709 12:14:01.873831 26370 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.853497 23682 rpcz_store.cc:276] 0709 12:13:52.390048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.390387 (+   339us) service_pool.cc:224] Handling call
0709 12:14:01.842705 (+9452318us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:58.049255 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:58.081393 (+ 32138us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:58.081924 (+   531us) write_op.cc:435] Acquired schema lock
0709 12:13:58.082924 (+  1000us) tablet.cc:662] Decoding operations
0709 12:13:58.202616 (+119692us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:58.202675 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:58.202701 (+    26us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:13:58.203242 (+   541us) tablet.cc:701] Row locks acquired
0709 12:13:58.203262 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:13:58.203434 (+   172us) write_op.cc:270] Start()
0709 12:13:58.203559 (+   125us) write_op.cc:276] Timestamp: P: 1783599238203375 usec, L: 0
0709 12:13:58.203584 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:13:58.240775 (+ 37191us) log.cc:844] Serialized 2430 byte log entry
0709 12:13:58.483831 (+243056us) op_driver.cc:464] REPLICATION: finished
0709 12:13:59.066602 (+582771us) write_op.cc:301] APPLY: starting
0709 12:13:59.066681 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:59.066849 (+   168us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:59.072329 (+  5480us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:59.073172 (+   843us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:59.073372 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:13:59.073421 (+    49us) write_op.cc:312] APPLY: finished
0709 12:13:59.073695 (+   274us) log.cc:844] Serialized 53 byte log entry
0709 12:13:59.073884 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:59.074010 (+   126us) write_op.cc:454] Released schema lock
0709 12:13:59.074051 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:59.074420 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:59.074446 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7710,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":582442,"apply.run_cpu_time_us":2736,"apply.run_wall_time_us":7791,"num_ops":6,"prepare.queue_time_us":172512,"prepare.run_cpu_time_us":5189,"prepare.run_wall_time_us":203294,"replication_time_us":258044,"thread_start_us":300544,"threads_started":2}]]}
I20260709 12:14:01.871749 26329 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:01.930637 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.928733 23785 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:01.878989 26436 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:01.886925 26399 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:01.941071 23851 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:01.874436 26459 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.873598 26420 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:01.944675 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:01.947515 23855 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:01.920187 23683 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:01.950511 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:01.950870 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:01.950845 23824 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:01.954202 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.952553 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:01.955570 23858 raft_consensus.cc:2410] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:14:01.952199 23857 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:01.956845 23853 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:01.887909 26377 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.957437 26377 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.957849 26377 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.962092 23683 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:01.963762 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:01.911368 26382 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:01.926920 26430 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.968448 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 138731us
I20260709 12:14:01.965894 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:01.966678 26437 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.927855 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.942130 26454 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:01.947748 26414 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:01.943555 26434 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.948321 26306 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.946863 26338 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:14:01.954447 26376 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:01.990538 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.069706 26376 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.978312 26459 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.070170 23932 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:01.927187 26415 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:14:01.962993 26393 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:01.887722 26337 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.977128 26297 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:01.993366 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.622324 23782 rpcz_store.cc:276] 0709 12:13:59.970393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.801098 (+830705us) service_pool.cc:224] Handling call
0709 12:14:00.821895 (+ 20797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.822118 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:01.122924 (+300806us) log.cc:844] Serialized 14 byte log entry
0709 12:14:01.607381 (+484457us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:01.607445 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:01.607489 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.607558 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:01.607710 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343038,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":1810,"raft.queue_time_us":84771,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5677,"thread_start_us":698364,"threads_started":2,"wal-append.queue_time_us":435219}
I20260709 12:14:01.991489 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:01.983809 26430 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.942490 26364 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:14:01.948465 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:01.969405 26161 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:02.065351 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:01.973188 23683 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 6, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 6 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:01.983716 26417 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:02.086570 26427 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:14:02.087352 25967 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:02.106519 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:02.082015 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 88743.5us
I20260709 12:14:01.961971 26320 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 12:14:02.094708 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.097193 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.100335 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:01.985641 26454 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:01.965418 26435 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.113s
I20260709 12:14:02.070237 26376 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:02.076373 26306 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.079300 26344 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.110375 23929 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:01.944077 26329 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:01.991041 26246 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:02.081004 26420 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:14:02.113513 23925 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.109476 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:02.452900 26435 consensus_queue.cc:1243] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:14:02.205304 26461 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:14:02.395941 23857 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:02.359208 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:02.140533 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.141563 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:02.167325 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.114017 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.178138 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:02.457894 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.142326 25174 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
W20260709 12:14:02.458307 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.458436 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.459333 23783 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:02.459749 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.124251 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.461226 23780 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:02.197384 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:02.355638 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:02.462527 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 265253us
I20260709 12:14:02.394511 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.463383 23858 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:02.394481 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.464430 23785 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:02.464461 23860 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:02.114043 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.393978 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:02.465551 25114 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:02.466370 23931 raft_consensus.cc:2376] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6. Re-sending same reply.
I20260709 12:14:02.113837 26434 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.455633 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 346190us
W20260709 12:14:02.145445 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49264.3us
I20260709 12:14:02.469272 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.466071 23928 raft_consensus.cc:2410] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:02.471365 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.454962 26454 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:02.472472 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:02.494874 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.472874 26398 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:02.202066 25173 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:02.344867 26320 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.298592 26377 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.246784 26405 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.214257 26337 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.476934 26340 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:02.477640 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.478271 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.181636 26414 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:02.478870 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.543705 23784 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.479480 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:02.475428 26462 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.357s	user 0.009s	sys 0.000s
I20260709 12:14:02.492251 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.494975 23857 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:02.453519 26435 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:02.495689 23925 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:02.496236 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.125972 26418 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.398s
I20260709 12:14:02.497421 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:02.182055 26465 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 12:14:02.201121 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.093113 26161 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:02.105736 26417 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.585985 26417 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:02.500399 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.501288 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:02.502274 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.026s	sys 0.045s
I20260709 12:14:02.200418 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.112762 26399 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:02.588857 23928 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.512887 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:02.589555 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76734.1us
I20260709 12:14:02.514325 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.512416 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.209520 26430 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.585909 26161 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:02.116417 26456 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:02.079020 26443 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.517407 23783 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:02.177870 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.518301 23854 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:02.162945 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.518682 26226 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:02.519846 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.459864 26306 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.347944 26373 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.523068 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.343439 26406 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:02.215411 26459 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.119993 26259 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:02.476648 26407 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:02.613112 23785 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:02.524910 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.613925 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.544548 23779 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:02.496810 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.092705 26393 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.432s
I20260709 12:14:02.567782 23929 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:02.615998 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.312997 26410 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.616588 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:02.584367 26480 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:02.584470 26337 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.523774 26320 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.294183 26372 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:02.587698 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.026s	sys 0.045s
W20260709 12:14:02.685467 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 731360us
I20260709 12:14:02.425994 26383 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.525775 26340 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:02.590478 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:02.319428 26423 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.595794 26351 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:02.698956 26351 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.699432 26351 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.699779 26437 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:02.700459 26389 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:02.701313 26389 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:02.701252 26437 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:02.701743 26389 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:02.702636 26389 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.705130 26389 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:02.706041 26372 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.500102 26434 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.713584 23926 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.601563 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:02.601449 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.602885 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 108709us
I20260709 12:14:02.717428 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:02.603996 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.555167 26435 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:02.608202 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.725710 23783 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:02.609411 23777 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:02.726534 23856 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:14:02.726922 23856 pending_rounds.cc:85] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 5
I20260709 12:14:02.727137 23856 pending_rounds.cc:107] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:14:02.544667 23781 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:02.614706 23932 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 3: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.616748 23784 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:14:02.320436 26370 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.618938 23927 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.616796 23929 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:02.629438 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.727255 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.729820 23777 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:02.730726 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.735630 26370 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.740494 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.741215 23781 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:02.741880 26417 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.745492 26434 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.746846 26297 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:02.747399 26297 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.748492 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:02.749486 26297 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.749672 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:02.750357 26435 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.750874 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:02.751683 26435 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:02.627758 26407 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:02.606796 26246 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:02.684897 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.765113 23931 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:02.684607 26449 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:02.735299 26410 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.694393 23930 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:02.775741 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:02.694348 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 75494.8us
W20260709 12:14:02.696327 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 105914us
I20260709 12:14:02.690392 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.684860 26337 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.718314 23782 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:02.727360 23856 raft_consensus.cc:2889] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:02.752804 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.727751 26406 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:02.754688 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.769445 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.757077 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:02.617285 26259 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.728222 26383 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.788475 26417 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.781648 26463 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:02.768200 26329 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:02.765923 26434 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:02.789770 23780 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:02.790903 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:02.748843 26377 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.753492 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.780373 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:02.772100 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54760.9us
I20260709 12:14:02.793342 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:02.783711 23783 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:02.683163 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.789106 26259 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.713699 26340 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.792454 23784 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:02.789803 23926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 196) took 1282 ms (client timeout 1648 ms). Trace:
I20260709 12:14:02.792531 23782 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:02.779191 26430 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:02.798750 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.821695 23777 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:02.822443 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:02.822821 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:02.815516 23926 rpcz_store.cc:269] 0709 12:14:01.507169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.507431 (+   262us) service_pool.cc:224] Handling call
0709 12:14:02.789601 (+1282170us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540032a854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:02.789739 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510579072}
I20260709 12:14:02.813651 23859 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:02.766840 26400 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.781737 25171 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:02.768240 26476 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.816932 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.818590 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.820303 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:02.778180 26398 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.782274 26382 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:02.786739 26344 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:02.789222 26383 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.866200 23782 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:02.725881 26423 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.840207 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.867460 23932 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:02.848223 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:02.840438 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 205711us
I20260709 12:14:02.840278 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.829118 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:02.841470 23930 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:02.842288 26400 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.828177 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.821585 26381 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:02.823526 23777 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:02.875196 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.827884 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.876535 23860 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:02.876579 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 47551.8us
I20260709 12:14:02.826894 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:02.877128 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 56914.1us
I20260709 12:14:02.828796 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.875892 23777 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 2.
I20260709 12:14:02.880256 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.868011 26377 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:02.881662 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.882319 23777 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:02.883201 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.883821 26456 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:02.885092 26373 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:02.823761 26475 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:14:02.886421 26475 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:02.886740 26475 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:02.887570 26475 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.888795 26475 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:02.889837 26329 consensus_queue.cc:1175] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ee4ab0747914e7abc642312517b5e25" responder_term: 6 status { last_received { term: 6 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 6 index: 5 } }
I20260709 12:14:02.892725 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:02.879621 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:02.894671 23782 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:02.843509 23780 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:02.839066 25967 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:02.828497 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.844578 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.790021 26226 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:02.844278 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 67443.5us
I20260709 12:14:02.767410 26441 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:14:02.798631 26297 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:02.864897 26344 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.814424 26406 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:02.797256 26393 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:02.781594 26362 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:14:02.866537 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.868906 23783 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:02.836438 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:02.868347 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.848637 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:02.869920 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:02.870332 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 93502.5us
I20260709 12:14:02.868643 23776 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:02.873371 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:02.898447 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:02.910732 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.850970 23784 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:02.898510 23777 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:02.913678 23751 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:02.845683 26417 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:02.819569 26434 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.925642 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:02.898939 26437 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:02.927532 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:02.979904 23926 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:02.980438 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 82063.4us
I20260709 12:14:02.912323 26226 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:02.910871 26430 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.984719 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:02.910387 26489 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:14:02.986497 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72905us
I20260709 12:14:02.870184 26407 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.010033 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:02.880093 26381 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.899086 26370 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:02.927198 26389 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:02.928014 26344 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:02.995205 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:02.996090 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:02.988591 26259 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:02.999115 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.027611 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.876785 26398 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.878578 26484 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:14:03.007701 23779 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:03.008453 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49014.1us
I20260709 12:14:02.958782 26417 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.011252 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:03.013125 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:02.932286 26393 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:02.911326 26340 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.980792 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:03.018738 26370 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:02.923864 26372 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:14:02.983879 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:02.998317 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 162451us
I20260709 12:14:02.993410 26399 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:03.023331 26362 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:03.026204 26489 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:03.033370 23783 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:02.876914 26476 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.013622 26437 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.028764 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:03.022720 26376 consensus_meta.cc:287] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
W20260709 12:14:03.028249 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.018471 26381 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:02.879186 26246 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:02.993090 26441 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:14:03.048089 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:03.020699 26320 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:03.022343 26297 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:03.048975 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 123413us
W20260709 12:14:03.057293 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 129826us
I20260709 12:14:03.049150 26463 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:03.077175 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:02.999732 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 151584us
I20260709 12:14:03.078444 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:03.079347 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 81768.5us
I20260709 12:14:03.036342 26329 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:03.040570 23930 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:03.014380 26430 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.043501 26389 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.056480 26399 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:03.034907 26417 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.044854 26337 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.079254 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:03.066175 26382 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.073989 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.062228 26437 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.058281 26362 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:03.063184 26259 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.087745 23783 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:03.086405 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:03.080592 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.091533 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:03.094615 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.096565 23932 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.096629 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.076844 26297 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.097851 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.100158 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.102178 26456 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:03.092144 25168 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:03.083287 26329 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.118966 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:03.098506 26405 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:03.141880 26498 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:03.142572 26405 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:03.141852 26503 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:03.144395 26503 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.144110 26405 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.087733 26499 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.089898 26399 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:03.150745 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:03.079839 26373 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:14:03.066915 26161 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:03.149449 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49363.9us
W20260709 12:14:03.154173 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62724.8us
I20260709 12:14:03.092825 26259 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:03.060702 26406 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.173000 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.073364 26376 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.149927 26337 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:03.087158 26320 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:14:03.167768 23932 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:03.149322 26329 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:03.098332 26297 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.163559 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:03.186988 23925 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:03.176704 23858 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:03.180346 26393 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:03.180804 26376 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:03.187706 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.193084 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.196780 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:03.174476 26488 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:14:03.097096 26382 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.210706 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:03.213064 23925 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:03.221969 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:03.180806 26259 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.166229 26500 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:03.163796 26501 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:03.241137 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.247538 23782 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:03.276918 23782 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:03.103986 26370 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.205055 26434 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.205452 26456 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:03.211807 26503 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:03.241400 26441 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:03.205250 26502 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.292194 23925 raft_consensus.cc:2410] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
W20260709 12:14:03.292888 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 198310us
W20260709 12:14:03.292467 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 281264us
I20260709 12:14:03.280536 26500 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.310865 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.164005 26499 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.280372 26389 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.326689 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:03.318794 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145868us
I20260709 12:14:03.319933 23932 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:03.318789 23933 raft_consensus.cc:1240] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:03.278807 26501 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.329077 23932 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:03.323568 23925 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:14:03.329752 23925 pending_rounds.cc:85] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 3
I20260709 12:14:03.329926 23925 pending_rounds.cc:107] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:14:03.330134 23925 raft_consensus.cc:2889] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:03.331642 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.333319 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.335439 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.334190 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:03.338639 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 274705us
I20260709 12:14:03.339572 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:03.343446 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 279515us
I20260709 12:14:03.284425 26506 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.306417 26503 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.165643 26489 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.344892 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:03.317492 26382 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:03.332062 26414 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.638s	user 0.005s	sys 0.006s
I20260709 12:14:03.377403 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:03.307508 26434 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:14:03.324940 26500 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.343036 26441 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:03.337128 26501 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.347862 26376 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:03.396577 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.349586 26320 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:03.352736 26340 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.338063 26490 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:03.384931 26373 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.343684 26425 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:14:03.376417 26503 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.387871 26476 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:03.397075 26441 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.347746 26513 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:03.401047 26506 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.376286 26512 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.403029 26430 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:03.404399 26320 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.404476 26376 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:03.429838 26454 consensus_meta.cc:287] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.897s	user 0.011s	sys 0.002s
I20260709 12:14:03.451742 26518 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:03.454182 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 76852.4us
I20260709 12:14:03.474566 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:03.420796 26373 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:03.461218 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 127991us
I20260709 12:14:03.438042 26441 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.475914 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.498167 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:03.483345 26351 consensus_meta.cc:287] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.783s	user 0.006s	sys 0.014s
W20260709 12:14:03.483461 26306 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.860s	user 0.006s	sys 0.006s
I20260709 12:14:03.483614 26373 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.515002 26494 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.487636 26454 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.489511 26437 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.435575 26513 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.477545 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:03.551470 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73990us
I20260709 12:14:03.552663 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.503279 23779 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:03.504772 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:03.442770 26512 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.460824 26320 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:03.406726 26504 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:14:03.515048 26306 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.515827 26494 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.473580 26376 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.474664 26476 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:03.480000 26518 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.550380 26423 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.587934 26382 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:03.443789 26508 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 12:14:03.572413 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.593777 25174 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
W20260709 12:14:03.598524 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 93820.3us
I20260709 12:14:03.556731 26512 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:03.603150 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 223670us
I20260709 12:14:03.623842 23900 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.427899 26490 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.593367 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:03.499917 26501 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.588045 26406 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:03.533519 26351 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.556303 26513 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.613554 23777 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:03.605960 26494 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:03.607857 26476 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.647979 23776 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:03.652931 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2274) took 1025 ms. Trace:
I20260709 12:14:03.655086 23777 rpcz_store.cc:276] 0709 12:14:02.635374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.613024 (+977650us) service_pool.cc:224] Handling call
0709 12:14:03.613368 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.650270 (+ 36902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.652869 (+  2599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:03.615768 26518 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:03.657001 23777 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:03.646626 25175 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:03.610227 26405 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.643527 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:03.621744 26382 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:03.637904 26406 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:03.659121 23777 raft_consensus.cc:1240] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 12:14:03.617966 26456 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:03.632213 26430 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.653620 26494 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.669639 23777 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 12:14:03.643342 26414 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:03.658964 26423 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.650475 26161 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.670682 23776 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:03.656373 26476 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:03.664201 26406 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:03.652861 25172 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.015s	sys 0.039s
I20260709 12:14:03.660070 26499 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:03.678647 23777 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:03.664824 26382 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:03.690649 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:03.697309 23932 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:03.679867 26430 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:03.681357 26514 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:14:03.692139 26435 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.935s	user 0.016s	sys 0.004s
I20260709 12:14:03.681180 26423 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:03.702689 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:03.703420 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:03.697173 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:03.666316 26405 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.667698 26456 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:03.755983 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:03.686566 26377 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:03.785135 23930 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:03.692595 25172 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.015s	sys 0.040s
I20260709 12:14:03.673084 26531 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:03.785274 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2218) took 1075 ms. Trace:
I20260709 12:14:03.672719 26506 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:14:03.703483 26501 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:03.679831 26524 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.695643 26519 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:03.790076 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.792272 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:03.805024 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:03.805787 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.810065 23776 rpcz_store.cc:276] 0709 12:14:02.709464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.694399 (+984935us) service_pool.cc:224] Handling call
0709 12:14:03.784809 (+ 90410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.784975 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.785000 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:03.785040 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:03.785067 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.785130 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.785221 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:03.889914 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.903486 23855 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:03.892370 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.915212 23857 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:03.860226 26521 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:14:03.938472 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:03.947983 26534 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.948462 26524 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.950110 26437 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.782521 26430 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.808441 26520 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.954615 23860 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.814009 26454 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.953510 26534 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.791121 26377 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:03.980108 26414 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:03.980543 26351 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:03.983103 23857 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4. Ops: []
W20260709 12:14:03.983552 25171 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 636) took 2110 ms (client timeout 2127 ms). Trace:
I20260709 12:14:04.001453 23855 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:03.894308 26531 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:03.958530 26306 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:03.787917 26510 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.403s
I20260709 12:14:04.006016 26501 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:04.020851 26525 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 12:14:04.007110 26437 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:04.025382 26543 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.025463 26437 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.026082 26543 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.012832 26405 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.021476 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:03.992269 26519 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:04.035550 23855 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:04.028628 26437 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.028635 26543 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:04.036569 26475 consensus_meta.cc:287] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.147s	user 0.007s	sys 0.023s
I20260709 12:14:04.048821 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:04.044157 26381 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.964s	user 0.016s	sys 0.006s
I20260709 12:14:04.012440 26520 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.059006 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 254965us
I20260709 12:14:04.037147 26528 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:04.092540 23854 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:04.015758 25171 rpcz_store.cc:269] 0709 12:14:01.872680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.874153 (+  1473us) service_pool.cc:224] Handling call
0709 12:14:03.983249 (+2109096us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b540064cdd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:03.983454 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384209152}
I20260709 12:14:04.030027 26501 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.126950 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 78225.6us
W20260709 12:14:04.150086 26527 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.361s	user 0.000s	sys 0.001s
I20260709 12:14:04.120501 26381 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.152783 26381 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.051141 26510 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:04.158871 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.125547 26504 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.148902 26437 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:04.169234 26532 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 12:14:04.138425 26417 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.035s	user 0.016s	sys 0.012s
I20260709 12:14:04.173398 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 273) took 1300 ms. Trace:
I20260709 12:14:04.120832 26537 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:04.173987 25177 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:14:04.173300 26344 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.139s	user 0.019s	sys 0.026s
I20260709 12:14:04.156104 26538 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:04.126287 26414 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.176079 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 86516.6us
I20260709 12:14:04.229764 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:04.326464 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:04.310909 23925 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.005s
W20260709 12:14:04.327996 23925 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.005s
I20260709 12:14:04.328354 23932 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:04.176435 23784 rpcz_store.cc:276] 0709 12:14:02.873188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.172614 (+1299426us) service_pool.cc:224] Handling call
0709 12:14:04.172935 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.173093 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.173115 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.173154 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:04.173183 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.173233 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.173341 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.323956 23860 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:04.147923 26542 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.136322 26528 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:04.325279 26540 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:14:04.333946 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:04.282246 26399 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.117s	user 0.008s	sys 0.003s
W20260709 12:14:04.279006 26329 consensus_meta.cc:287] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.067s	user 0.012s	sys 0.016s
W20260709 12:14:04.280256 26297 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.068s	user 0.011s	sys 0.000s
I20260709 12:14:04.169935 26456 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.170683 26437 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:04.280531 26377 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.327718 26344 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.340085 23860 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:04.355677 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.211969 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:04.182318 26405 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:04.165076 26510 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.350477 26534 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.159996 26531 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.368156 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:04.342177 26400 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.448s	user 0.028s	sys 0.021s
W20260709 12:14:04.341823 26382 consensus_meta.cc:287] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.638s	user 0.029s	sys 0.022s
W20260709 12:14:04.341799 26226 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.326s	user 0.002s	sys 0.023s
I20260709 12:14:04.334229 26538 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.331631 26537 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.339623 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:04.196805 26417 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.360088 26533 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:14:04.360842 26510 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.361281 26528 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.345364 26329 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.341910 26399 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.365252 26406 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.672s	user 0.015s	sys 0.011s
W20260709 12:14:04.365494 26259 consensus_meta.cc:287] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.084s	user 0.018s	sys 0.012s
I20260709 12:14:04.367252 26344 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.353271 26437 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.374549 23932 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:04.364699 26430 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.388854 23856 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:04.389350 23856 raft_consensus.cc:614] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:04.389897 23855 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:04.348685 26297 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.373032 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.385753 26259 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.392545 26443 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.392586 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 425) took 1014 ms. Trace:
I20260709 12:14:04.174428 26504 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.396881 23860 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:04.399552 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.400209 23860 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:04.382941 26501 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.400568 23932 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:04.401311 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.381846 26524 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.404568 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:04.405498 23860 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:04.373980 26400 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.406126 23778 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:04.166133 26519 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.409350 23780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2262) took 1938 ms (client timeout 2126 ms). Trace:
I20260709 12:14:04.379262 26382 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.410495 23780 rpcz_store.cc:269] 0709 12:14:02.470954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.491993 (+ 21039us) service_pool.cc:224] Handling call
0709 12:14:04.409166 (+1917173us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400166f54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:04.409285 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548899584}
I20260709 12:14:04.412422 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.372246 26405 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.421079 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 275) took 1462 ms. Trace:
I20260709 12:14:04.422340 23780 rpcz_store.cc:276] 0709 12:14:02.958828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.420196 (+1461368us) service_pool.cc:224] Handling call
0709 12:14:04.420550 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.420733 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.420770 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.420815 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:04.420846 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.420897 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.421013 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:04.405035 26373 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.873s	user 0.008s	sys 0.037s
I20260709 12:14:04.386160 26406 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.381940 26226 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.427028 26329 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.398727 25171 rpcz_store.cc:276] 0709 12:14:03.377968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.166258 (+788290us) service_pool.cc:224] Handling call
0709 12:14:04.392486 (+226228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.435940 23851 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:04.436558 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:04.437703 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 422) took 1088 ms. Trace:
I20260709 12:14:04.438161 25171 rpcz_store.cc:276] 0709 12:14:03.348916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.436162 (+1087246us) service_pool.cc:224] Handling call
0709 12:14:04.437623 (+  1461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.440477 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 642) took 1544 ms. Trace:
I20260709 12:14:04.440838 25174 rpcz_store.cc:276] 0709 12:14:02.896509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.897693 (+  1184us) service_pool.cc:224] Handling call
0709 12:14:02.929324 (+ 31631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.619167 (+689843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.440258 (+821091us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54006d0e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:04.440417 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524646656}
I20260709 12:14:04.417763 26306 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:04.345311 26542 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.460526 26297 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.468401 25176 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:14:04.436599 26495 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 1.336s	user 0.007s	sys 0.000s
I20260709 12:14:04.503356 23931 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:04.504474 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:04.508311 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 277) took 1522 ms. Trace:
I20260709 12:14:04.543665 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:04.556259 23781 rpcz_store.cc:276] 0709 12:14:02.986049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.507450 (+1521401us) service_pool.cc:224] Handling call
0709 12:14:04.507777 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.507967 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.507994 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.508038 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:04.508069 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.508120 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.508234 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.439541 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:04.560580 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 424) took 1197 ms. Trace:
I20260709 12:14:04.563634 25174 rpcz_store.cc:276] 0709 12:14:03.363709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.441880 (+1078171us) service_pool.cc:224] Handling call
0709 12:14:04.560480 (+118600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.564862 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:04.565658 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 423) took 1216 ms. Trace:
I20260709 12:14:04.566035 25174 rpcz_store.cc:276] 0709 12:14:03.349186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.564459 (+1215273us) service_pool.cc:224] Handling call
0709 12:14:04.565598 (+  1139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.567011 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:04.450448 26400 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.447993 26443 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:04.568409 26443 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.568662 26405 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:04.573709 26443 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.575671 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 429) took 1126 ms. Trace:
I20260709 12:14:04.577211 25174 rpcz_store.cc:276] 0709 12:14:03.449060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.566638 (+1117578us) service_pool.cc:224] Handling call
0709 12:14:04.575566 (+  8928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.582592 26226 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.430320 26373 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.641024 26417 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.926590 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:04.981452 26451 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:04.826954 25171 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 426) took 1447 ms (client timeout 1609 ms). Trace:
I20260709 12:14:04.507333 26306 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:04.982270 26451 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:04.952258 23854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 929) took 2123 ms (client timeout 1666 ms). Trace:
I20260709 12:14:04.888576 26297 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:04.966787 26399 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:04.882747 26518 consensus_meta.cc:287] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.216s	user 0.010s	sys 0.020s
I20260709 12:14:04.984520 26399 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.984699 26518 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:14:04.951381 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2270) took 2346 ms (client timeout 1738 ms). Trace:
I20260709 12:14:04.985042 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:04.985703 26451 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:04.986598 23931 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:04.983182 23854 rpcz_store.cc:269] 0709 12:14:02.828786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.828890 (+   104us) service_pool.cc:224] Handling call
0709 12:14:04.952036 (+2123146us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004137d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:04.952151 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462407424}
I20260709 12:14:04.987387 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 432) took 1537 ms. Trace:
I20260709 12:14:04.987852 25174 rpcz_store.cc:276] 0709 12:14:03.450294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.579101 (+1128807us) service_pool.cc:224] Handling call
0709 12:14:04.987323 (+408222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:04.757169 26513 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.099s	user 0.014s	sys 0.006s
W20260709 12:14:04.683329 26441 consensus_meta.cc:287] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.151s	user 0.022s	sys 0.048s
I20260709 12:14:04.640563 26443 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.941651 26362 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.937489 26382 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:04.993496 26520 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:04.998504 26382 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:04.999197 26382 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.912290 26437 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.000s
I20260709 12:14:04.990537 26441 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:05.001523 26382 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.001271 26437 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.000583 26393 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:05.002709 26393 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:04.989643 26513 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.756234 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2217) took 1925 ms. Trace:
I20260709 12:14:05.003501 26393 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:04.744899 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.990836 26443 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:14:04.635003 26320 consensus_meta.cc:287] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.035s	user 0.007s	sys 0.024s
I20260709 12:14:05.005407 26320 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:05.006174 26443 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.006424 23853 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:04.991669 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.007197 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 427) took 1604 ms. Trace:
I20260709 12:14:05.007870 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2461) took 1214 ms. Trace:
I20260709 12:14:05.008294 23853 rpcz_store.cc:276] 0709 12:14:03.793132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.793272 (+   140us) service_pool.cc:224] Handling call
0709 12:14:03.817219 (+ 23947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.006856 (+1189637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.007510 (+   654us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002df854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.007785 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2734496640}
I20260709 12:14:05.008883 26491 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:05.009287 26491 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:05.009516 26513 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.009696 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.010056 26491 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.010850 23933 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:05.010869 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:04.993135 26373 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:04.957203 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 238282us
I20260709 12:14:05.007514 25174 rpcz_store.cc:276] 0709 12:14:03.403288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.989056 (+1585768us) service_pool.cc:224] Handling call
0709 12:14:05.007133 (+ 18077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.928604 26537 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:04.746024 26553 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 12:14:05.014008 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:04.857635 26512 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.209s	user 0.018s	sys 0.014s
I20260709 12:14:05.018628 26512 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:04.639559 26494 consensus_meta.cc:287] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.959s	user 0.015s	sys 0.014s
W20260709 12:14:04.640623 26503 consensus_meta.cc:287] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.200s	user 0.025s	sys 0.010s
I20260709 12:14:04.931224 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:04.995916 26351 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:04.928313 26259 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:04.995690 23779 rpcz_store.cc:269] 0709 12:14:02.605056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.624731 (+ 19675us) service_pool.cc:224] Handling call
0709 12:14:04.951095 (+2326364us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400168354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:04.951259 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530254080}
I20260709 12:14:04.747961 26538 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.004711 26393 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.987787 26399 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:04.954927 26504 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.012790 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.011046 26491 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:04.638054 26376 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.005s	user 0.020s	sys 0.024s
I20260709 12:14:05.014621 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2220) took 2304 ms. Trace:
I20260709 12:14:04.503185 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 645) took 1428 ms. Trace:
W20260709 12:14:04.935326 26476 consensus_meta.cc:287] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.239s	user 0.007s	sys 0.027s
I20260709 12:14:05.016644 23852 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:05.016850 26441 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.044391 26351 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:04.589398 26406 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.070675 26399 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.902926 26542 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:04.976878 23932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.018s	sys 0.030s
I20260709 12:14:05.071280 26399 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:04.759863 26500 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.341s	user 0.012s	sys 0.001s
W20260709 12:14:05.019881 25171 rpcz_store.cc:269] 0709 12:14:03.380083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.439163 (+1059080us) service_pool.cc:224] Handling call
0709 12:14:04.816338 (+377175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.998518 26362 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:05.014773 25174 raft_consensus.cc:2368] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:05.021322 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:05.140770 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 119518us
I20260709 12:14:05.141498 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:04.984123 26297 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.143678 26443 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:05.143940 26259 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:05.025643 26554 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.632s	user 0.001s	sys 0.006s
I20260709 12:14:05.136474 26399 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.145771 26566 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.146390 26399 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.146284 26566 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.148375 26566 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:04.983259 26306 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.140025 26500 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.151278 26500 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.130921 26406 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.152017 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 210) took 1464 ms. Trace:
I20260709 12:14:05.004043 23929 rpcz_store.cc:276] 0709 12:14:02.830668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.877790 (+ 47122us) service_pool.cc:224] Handling call
0709 12:14:02.878149 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:02.891984 (+ 13835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.892015 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:02.892603 (+   588us) log.cc:844] Serialized 52 byte log entry
0709 12:14:03.041598 (+148995us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:03.041640 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.041713 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.413441 (+1371728us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:04.755873 (+342432us) raft_consensus.cc:1685] finished
0709 12:14:04.755899 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.756171 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13367,"raft.queue_time_us":14652,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":19868032,"thread_start_us":148572,"threads_started":1,"wal-append.queue_time_us":198596}
I20260709 12:14:05.030730 26498 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.154598 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.033812 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.034294 26494 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.042290 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.042637 26503 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.043320 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:05.141434 26519 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.015s	sys 0.023s
I20260709 12:14:05.053683 25170 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:05.043807 26415 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:05.057528 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.058457 26459 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:05.062949 23780 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:05.161281 23898 connection.cc:256] RPC call timeout handler was delayed by -1.2107673390000855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 12:14:05.162725 26415 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.063923 23776 rpcz_store.cc:276] 0709 12:14:02.709935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.885715 (+1175780us) service_pool.cc:224] Handling call
0709 12:14:03.896653 (+ 10938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.023929 (+127276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.023962 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.043382 (+ 19420us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.339711 (+296329us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:04.339759 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.339826 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.014425 (+674599us) raft_consensus.cc:1685] finished
0709 12:14:05.014446 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.014565 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5236,"raft.queue_time_us":86109,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3825,"thread_start_us":266459,"threads_started":2,"wal-append.queue_time_us":314001}
W20260709 12:14:05.071529 23932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.018s	sys 0.030s
I20260709 12:14:05.007524 26320 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:05.017980 23931 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:05.059379 26226 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:14:05.080050 26476 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.164796 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.165313 26494 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.165580 23931 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:05.065855 25173 rpcz_store.cc:276] 0709 12:14:03.038801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.150449 (+111648us) service_pool.cc:224] Handling call
0709 12:14:03.241198 (+ 90749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.311029 (+ 69831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.320223 (+  9194us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.389740 (+ 69517us) log.cc:844] Serialized 26 byte log entry
0709 12:14:03.648859 (+259119us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:03.654428 (+  5569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.658525 (+  4097us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.440219 (+781694us) raft_consensus.cc:1685] finished
0709 12:14:04.440241 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.467544 (+ 27303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12145,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8984,"thread_start_us":106327,"threads_started":1,"wal-append.queue_time_us":270194}
I20260709 12:14:05.070677 26351 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.162165 26459 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.166821 26351 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:05.066617 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 640) took 2165 ms. Trace:
I20260709 12:14:05.167557 26297 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.163532 23784 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:05.169086 26459 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.169323 26490 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:05.140698 25174 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 431) took 1690 ms (client timeout 2221 ms). Trace:
W20260709 12:14:05.169807 25174 rpcz_store.cc:269] 0709 12:14:03.449918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.013614 (+1563696us) service_pool.cc:224] Handling call
0709 12:14:05.140619 (+127005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.170169 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 274) took 2245 ms. Trace:
I20260709 12:14:05.028904 26364 consensus_queue.cc:1059] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:05.144235 26443 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.171082 26443 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:05.152205 26406 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.172087 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47418 (request call id 19) took 1552 ms. Trace:
I20260709 12:14:05.172464 23856 rpcz_store.cc:276] 0709 12:14:03.619893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.620014 (+   121us) service_pool.cc:224] Handling call
0709 12:14:05.171899 (+1551885us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400436cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7ac853 00007fb2b277a60a 00007fb2abd007d4 00007fb2abd00762 00007fb2abd00691 00007fb2abd0060c 00007fb2abcff8c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.172000 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679825280}
I20260709 12:14:05.173295 26406 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.173597 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.174353 23856 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.175104 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.153584 23859 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
W20260709 12:14:04.994588 26517 log.cc:927] Time spent T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 1.498s	user 0.000s	sys 0.008s
I20260709 12:14:05.036082 23928 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:05.068981 26437 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.152453 23925 rpcz_store.cc:276] 0709 12:14:03.687841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.687993 (+   152us) service_pool.cc:224] Handling call
0709 12:14:03.688250 (+   257us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.688555 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.688580 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.704458 (+ 15878us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.329203 (+624745us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:04.374254 (+ 45051us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.374965 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.375018 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.025046 (+650028us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:05.151685 (+126639us) raft_consensus.cc:1685] finished
0709 12:14:05.151786 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.151952 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5053,"prepare.queue_time_us":93945,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":10018,"raft.queue_time_us":133457,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":66194,"spinlock_wait_cycles":3411200,"thread_start_us":554221,"threads_started":2,"wal-append.queue_time_us":570807}
I20260709 12:14:05.155201 23929 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:05.043900 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.156626 23926 raft_consensus.cc:2368] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:05.130286 26502 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.182331 23860 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:05.166020 26476 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.184077 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.182338 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.159675 23854 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:05.160562 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.185755 23927 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:05.064245 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.163080 26500 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.080905 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.189404 23852 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.167781 25172 rpcz_store.cc:276] 0709 12:14:02.895936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.896654 (+   718us) service_pool.cc:224] Handling call
0709 12:14:02.896993 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.897208 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:03.068974 (+171766us) log.cc:844] Serialized 14 byte log entry
0709 12:14:05.043309 (+1974335us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b18002c8000. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a
0709 12:14:05.043526 (+   217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.043582 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:05.043683 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.044361 (+   678us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.059644 (+ 15283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2474,"prepare.queue_time_us":184533,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":3099,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4214,"spinlock_wait_cycles":38179584,"thread_start_us":887599,"threads_started":2,"wal-append.queue_time_us":718987}
I20260709 12:14:05.136083 26512 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.170796 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.190408 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.177011 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.177186 23856 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.068290 26376 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.178349 23824 leader_election.cc:400] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:05.191566 23928 raft_consensus.cc:2376] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5. Re-sending same reply.
I20260709 12:14:05.155519 26259 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.170583 23784 rpcz_store.cc:276] 0709 12:14:02.925083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.333112 (+1408029us) service_pool.cc:224] Handling call
0709 12:14:04.343214 (+ 10102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.168150 (+824936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.169952 (+  1802us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b540010c054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.170112 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886142208}
I20260709 12:14:05.178257 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:05.186635 23853 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:05.163228 26503 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.185124 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:05.159811 26519 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.015s	sys 0.023s
W20260709 12:14:05.156481 26570 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:14:05.178089 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.192505 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.193074 23854 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.178807 23898 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:05.193724 23824 leader_election.cc:403] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:14:05.194578 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.195124 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.183043 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.229467 23857 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.238716 23926 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:05.238878 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 60711.8us
I20260709 12:14:05.239344 23852 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.159626 26572 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.170212 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.159583 26498 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.241099 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:05.242029 26376 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.242719 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.197383 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.244130 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:05.300203 23856 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:05.244251 26364 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:05.244786 26459 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:05.245040 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.245653 26226 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:05.247164 26393 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:05.307933 26504 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:05.308465 26504 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:05.310452 26498 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:05.247629 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 68857.3us
I20260709 12:14:05.309664 26504 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.248169 26441 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:05.239444 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:05.249517 25175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 644) took 2232 ms (client timeout 1712 ms). Trace:
W20260709 12:14:05.243878 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 65666us
I20260709 12:14:05.252627 26463 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:05.253144 23853 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:05.320495 26459 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.327633 23853 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:05.253660 26454 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:05.255105 26451 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:05.243981 26518 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.183075 26566 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:05.331121 25175 rpcz_store.cc:269] 0709 12:14:03.017056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.032804 (+ 15748us) service_pool.cc:224] Handling call
0709 12:14:05.249324 (+2216520us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540067e8d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.249452 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308904832}
I20260709 12:14:05.244376 26490 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 12:14:05.259403 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:05.289865 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.356060 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 930) took 2257 ms. Trace:
I20260709 12:14:05.295030 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.357663 23926 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:05.252592 23929 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:05.262090 23927 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:05.260916 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.360504 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:05.278118 23749 connection.cc:256] RPC call timeout handler was delayed by -1.4951345730000867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 12:14:05.182476 26373 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.194406 26406 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.365123 23852 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:05.229123 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:05.306195 23854 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.294618 26405 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:05.312778 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.370616 26405 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.371306 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.372869 26405 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.323865 26568 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:14:05.374256 26568 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:05.374635 26568 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:05.375146 23926 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:05.314265 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.378793 23860 raft_consensus.cc:2410] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:05.198261 26259 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:05.193316 26502 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:05.327628 23934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.009s	sys 0.011s
W20260709 12:14:05.379940 23934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.011s
I20260709 12:14:05.380862 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2216) took 2555 ms. Trace:
I20260709 12:14:05.382081 26502 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.382495 26493 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:05.382706 26372 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:05.194041 26500 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:05.346487 23900 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.712s (SENT) [suppressed 25 similar messages]
I20260709 12:14:05.384589 26372 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:05.385187 26372 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:05.361181 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.387321 23860 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:05.388922 23925 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:05.389726 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:05.390404 23785 raft_consensus.cc:2376] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4. Re-sending same reply.
I20260709 12:14:05.391821 26518 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.392040 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.392704 23782 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:14:05.392499 26518 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.393666 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.332268 26463 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:05.395763 26389 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.396648 26398 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:05.396787 26435 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.398379 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.356674 23858 rpcz_store.cc:276] 0709 12:14:03.076347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.076544 (+   197us) service_pool.cc:224] Handling call
0709 12:14:03.175413 (+ 98869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.187982 (+ 12569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.333224 (+2145242us) spinlock_profiling.cc:243] Waited 86.2 ms on lock 0x7b5400025150. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.333356 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198251904}
W20260709 12:14:05.320044 26370 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.616s	user 0.005s	sys 0.021s
I20260709 12:14:05.302304 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.369462 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:05.403038 25175 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:05.314414 23856 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:05.376055 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.376173 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.404867 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.391460 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:05.405931 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.406358 23785 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:05.406513 23927 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:05.356832 26566 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.330483 26454 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.354825 25172 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:14:05.381201 23934 rpcz_store.cc:276] 0709 12:14:02.825750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.828937 (+  3187us) service_pool.cc:224] Handling call
0709 12:14:02.829274 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.829631 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.829660 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.846245 (+ 16585us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.703502 (+1857257us) spinlock_profiling.cc:243] Waited 8.98 ms on lock 0x7b1800215ec0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:14:05.380353 (+676851us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:05.380395 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.380474 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.380520 (+    46us) raft_consensus.cc:1685] finished
0709 12:14:05.380531 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.380756 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1975,"raft.queue_time_us":19689,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3762,"spinlock_wait_cycles":20644096,"thread_start_us":676649,"threads_started":1,"wal-append.queue_time_us":2184818}
I20260709 12:14:05.320863 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.383086 26493 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:05.386294 23854 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:05.354777 26376 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:14:05.398995 23783 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:05.421556 26376 consensus_queue.cc:1243] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:05.421896 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.423044 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.324091 26504 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:05.399914 23779 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:05.332010 23824 leader_election.cc:400] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
W20260709 12:14:05.402906 26370 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.699s	user 0.010s	sys 0.026s
I20260709 12:14:05.379195 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.428786 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.407450 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.430140 23927 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:05.413374 23931 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:05.413444 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.325412 26226 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:05.425421 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.425612 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.434552 23776 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:05.425742 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:05.414203 26502 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.435519 23783 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:05.426405 23781 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:05.426524 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:05.428297 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 96412.1us
I20260709 12:14:05.437121 23784 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:05.437335 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:05.361493 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.438026 23929 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:05.431490 23782 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:05.401968 26405 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.397585 26435 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.443051 26506 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:05.444247 26506 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:05.398005 26459 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.444657 26506 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:05.396483 26389 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.414463 26503 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.439261 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.451572 23779 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 2.
I20260709 12:14:05.439378 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:05.453299 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:05.438160 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:05.426862 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:05.440588 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.366757 26373 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.300084 26572 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.416115 26545 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.416683 26518 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.368273 26406 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.449256 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.448307 23785 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:05.454154 25175 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:05.446002 26566 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.455891 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:05.454584 26376 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:05.375492 26568 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.396569 26463 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.460326 23783 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:05.461328 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.462255 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:05.430683 26441 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:14:05.445457 26506 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.528152 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:05.810861 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 189) took 2880 ms. Trace:
I20260709 12:14:05.798324 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2273) took 3175 ms. Trace:
I20260709 12:14:05.938524 23926 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
W20260709 12:14:05.441668 25171 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 433) took 1627 ms (client timeout 2036 ms). Trace:
I20260709 12:14:05.939126 23928 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:05.822543 23859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 933) took 2445 ms (client timeout 1758 ms). Trace:
I20260709 12:14:05.716800 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:05.943051 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:05.810350 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.446600 26370 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:05.938055 23854 raft_consensus.cc:2410] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:05.937259 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.521436 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:05.804929 23933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 216) took 2114 ms (client timeout 2137 ms). Trace:
I20260709 12:14:05.964586 23785 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:05.809010 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2219) took 3099 ms. Trace:
I20260709 12:14:05.463380 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:05.956674 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.938091 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.814636 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.815621 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:05.964443 23779 raft_consensus.cc:2376] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
W20260709 12:14:05.946135 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 490243us
I20260709 12:14:05.595116 23934 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:05.816730 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.804239 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.967974 23781 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:05.965663 23777 rpcz_store.cc:276] 0709 12:14:02.709703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.695370 (+985667us) service_pool.cc:224] Handling call
0709 12:14:03.701368 (+  5998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.951110 (+249742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:03.951139 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.952196 (+  1057us) log.cc:844] Serialized 556 byte log entry
0709 12:14:04.129752 (+177556us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:04.129793 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.808613 (+1678820us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540010ac54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.808660 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.808708 (+    48us) raft_consensus.cc:1685] finished
0709 12:14:05.808720 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.808946 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260248448,"thread_start_us":177243,"threads_started":1,"wal-append.queue_time_us":276754,"wal-append.run_cpu_time_us":1749,"wal-append.run_wall_time_us":1027299}
W20260709 12:14:05.826972 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 2460) took 2040 ms (client timeout 1712 ms). Trace:
W20260709 12:14:05.965111 23933 rpcz_store.cc:269] 0709 12:14:03.690395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.693945 (+  3550us) service_pool.cc:224] Handling call
0709 12:14:05.804710 (+2110765us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002030d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.804839 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2822692096}
I20260709 12:14:05.813616 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:05.816042 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 276) took 2837 ms. Trace:
I20260709 12:14:05.973768 23925 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:05.967576 23782 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:05.968475 23929 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:14:05.825659 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 192) took 2894 ms. Trace:
I20260709 12:14:05.976071 23780 rpcz_store.cc:276] 0709 12:14:02.979167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.423013 (+1443846us) service_pool.cc:224] Handling call
0709 12:14:04.423268 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.162320 (+739052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.815810 (+653490us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400166554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.815981 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470643968}
W20260709 12:14:05.463666 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.977775 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:05.978662 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:05.979116 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:05.979463 23780 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:05.520056 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:05.556620 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 129830us
W20260709 12:14:05.527000 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.968473 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:05.971848 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:05.646492 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.972445 23930 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:05.760217 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 938) took 1728 ms. Trace:
I20260709 12:14:05.938843 23778 rpcz_store.cc:276] 0709 12:14:02.623174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.346509 (+723335us) service_pool.cc:224] Handling call
0709 12:14:03.346823 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.410369 (+1063546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.798043 (+1387674us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001ee454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.798249 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435731712}
W20260709 12:14:05.970517 23859 rpcz_store.cc:269] 0709 12:14:03.376922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.378147 (+  1225us) service_pool.cc:224] Handling call
0709 12:14:05.822314 (+2444167us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54002d4bd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.822460 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4029201792}
W20260709 12:14:05.972975 23851 rpcz_store.cc:269] 0709 12:14:03.786435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.804711 (+ 18276us) service_pool.cc:224] Handling call
0709 12:14:05.826731 (+2022020us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b5400024c54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.826867 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1249187200}
I20260709 12:14:05.824541 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:05.759487 26505 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 2.285s	user 0.007s	sys 0.009s
W20260709 12:14:05.521097 26576 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:14:05.814854 26443 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.000s
I20260709 12:14:05.975576 25176 rpcz_store.cc:276] 0709 12:14:02.931125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.931221 (+    96us) service_pool.cc:224] Handling call
0709 12:14:02.984364 (+ 53143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.553593 (+1569229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.825378 (+1271785us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001fac54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.825594 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3412241280}
I20260709 12:14:05.793613 26337 consensus_queue.cc:1059] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.192s
I20260709 12:14:05.444664 26435 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:05.988657 23780 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:05.990111 23931 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:05.990859 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:05.709067 26580 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:14:05.759343 26577 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.356s	user 0.001s	sys 0.000s
I20260709 12:14:05.983351 23855 rpcz_store.cc:276] 0709 12:14:04.031786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.054336 (+ 22550us) service_pool.cc:224] Handling call
0709 12:14:04.130256 (+ 75920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.390593 (+260337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.759952 (+1369359us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400436cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.760148 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596723968}
I20260709 12:14:05.799404 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:05.630584 25168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 181) took 3532 ms (client timeout 1520 ms). Trace:
I20260709 12:14:05.797709 26362 raft_consensus.cc:3055] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:06.015230 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:06.036193 23928 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:05.780802 26364 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.994781 26531 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:06.037981 23931 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:05.996027 23933 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:05.994350 23930 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:05.770179 26383 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:06.041455 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.536963 26476 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:05.940158 26519 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:06.043982 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:06.102816 23926 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:05.819921 26423 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.520392 26500 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:06.048035 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 225) took 1003 ms. Trace:
I20260709 12:14:06.110564 23925 rpcz_store.cc:276] 0709 12:14:05.045015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.986145 (+941130us) service_pool.cc:224] Handling call
0709 12:14:06.047473 (+ 61328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.047682 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.047703 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.047756 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:06.047797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.047849 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.047975 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.804957 26373 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.446990 26502 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.448282 26405 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.115747 26405 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.990477 26376 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:06.117358 26376 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.058862 26572 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:06.121476 26572 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:14:06.126708 26372 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.741s	user 0.005s	sys 0.006s
W20260709 12:14:06.106546 26579 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.428s	user 0.001s	sys 0.000s
W20260709 12:14:06.129305 26578 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.439s	user 0.000s	sys 0.001s
I20260709 12:14:06.054944 26389 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:06.051775 25168 rpcz_store.cc:269] 0709 12:14:02.097790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.107773 (+  9983us) service_pool.cc:224] Handling call
0709 12:14:05.630307 (+3522534us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400614254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:05.630499 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057539456}
I20260709 12:14:06.133941 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:06.056722 26506 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:05.814968 26523 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 2.112s	user 0.003s	sys 0.013s
I20260709 12:14:05.938114 25177 rpcz_store.cc:276] 0709 12:14:02.930418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.930534 (+   116us) service_pool.cc:224] Handling call
0709 12:14:02.989562 (+ 59028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.331198 (+1341636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.810622 (+1479424us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001e87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.810807 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2672430336}
I20260709 12:14:06.051142 25173 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:06.054190 26362 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:06.154289 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:05.467998 26393 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:05.526094 26382 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.517s
I20260709 12:14:05.463495 26493 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.815215 26454 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:05.994235 26491 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:06.068965 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.263819 25176 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:06.264408 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 435) took 1219 ms. Trace:
W20260709 12:14:05.965258 25171 rpcz_store.cc:269] 0709 12:14:03.814504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.182666 (+1368162us) service_pool.cc:224] Handling call
0709 12:14:05.440040 (+257374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.072863 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.036504 26441 raft_consensus.cc:2764] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:14:06.083458 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:06.041998 26306 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:05.460326 26503 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.090811 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.099810 23781 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:05.986487 26430 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 12:14:05.541028 26545 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:05.639015 25169 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:05.703750 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 639) took 3825 ms. Trace:
I20260709 12:14:05.939962 26512 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:05.980377 26370 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.994001 26484 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 12:14:06.113849 26373 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:05.524693 26498 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.567s
W20260709 12:14:06.178367 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.135296 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.331565 26362 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.349090 26362 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:06.349386 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:06.349552 26512 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.350682 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.127179 26372 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.352208 26512 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:06.272032 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 436) took 1218 ms. Trace:
I20260709 12:14:06.352905 25173 rpcz_store.cc:276] 0709 12:14:05.045609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.799084 (+753475us) service_pool.cc:224] Handling call
0709 12:14:06.263651 (+464567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.353348 26372 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:06.354732 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.335539 23776 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:06.355702 26430 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:06.358063 26512 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:06.358832 26512 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.360251 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:06.361052 25173 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:06.361907 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:06.362444 23776 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:06.184531 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 651) took 1029 ms. Trace:
I20260709 12:14:06.366021 25177 rpcz_store.cc:276] 0709 12:14:05.155568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.153891 (+998323us) service_pool.cc:224] Handling call
0709 12:14:06.184467 (+ 30576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.248200 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 891) took 1085 ms. Trace:
I20260709 12:14:06.184789 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.185411 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 226) took 1121 ms. Trace:
I20260709 12:14:06.264743 25176 rpcz_store.cc:276] 0709 12:14:05.045376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.047814 (+1002438us) service_pool.cc:224] Handling call
0709 12:14:06.264349 (+216535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.089275 23931 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:06.366953 23934 rpcz_store.cc:276] 0709 12:14:05.163075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.413209 (+250134us) service_pool.cc:224] Handling call
0709 12:14:06.247970 (+834761us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54003534d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.248119 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230148352}
I20260709 12:14:06.367933 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 899) took 1052 ms. Trace:
I20260709 12:14:06.265799 23785 raft_consensus.cc:2410] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:06.368319 23931 rpcz_store.cc:276] 0709 12:14:05.315058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.088779 (+773721us) service_pool.cc:224] Handling call
0709 12:14:06.089103 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.367642 (+278539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.367868 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.265760 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 479622us
I20260709 12:14:06.369648 23929 rpcz_store.cc:276] 0709 12:14:05.063620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.996148 (+932528us) service_pool.cc:224] Handling call
0709 12:14:05.996437 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.028149 (+ 31712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.028181 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.028222 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:06.028263 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.185196 (+156933us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b54002a5154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:06.185241 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.185353 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72445824}
I20260709 12:14:06.254911 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.271278 23859 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:06.283154 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.375126 23778 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:06.375692 25175 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:06.375769 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.376350 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 440) took 1329 ms. Trace:
I20260709 12:14:06.376504 23782 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:06.376750 25175 rpcz_store.cc:276] 0709 12:14:05.051122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.109614 (+1058492us) service_pool.cc:224] Handling call
0709 12:14:06.376286 (+266672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.377485 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.378588 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.284996 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.284111 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 647) took 1140 ms. Trace:
I20260709 12:14:06.379933 25174 rpcz_store.cc:276] 0709 12:14:05.143319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.045034 (+901715us) service_pool.cc:224] Handling call
0709 12:14:06.284042 (+239008us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.288967 25113 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.021s	sys 0.016s
W20260709 12:14:06.381011 25113 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.022s	sys 0.017s
W20260709 12:14:06.381327 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 597868us
I20260709 12:14:06.088560 26364 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:05.659180 26226 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.394815 23782 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:06.395520 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2287) took 1005 ms. Trace:
I20260709 12:14:06.395859 23782 rpcz_store.cc:276] 0709 12:14:05.389932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.394306 (+1004374us) service_pool.cc:224] Handling call
0709 12:14:06.394618 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.395285 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.395466 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.396912 23853 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:06.397380 23782 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:06.293171 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2248) took 1157 ms. Trace:
I20260709 12:14:06.087908 26443 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.271720 26494 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:14:06.400110 23927 rpcz_store.cc:276] 0709 12:14:05.135995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.028004 (+892009us) service_pool.cc:224] Handling call
0709 12:14:06.028365 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.104653 (+ 76288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.104690 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.104749 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:06.104787 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.292951 (+188164us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b54002035d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:06.292996 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.293114 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174936960}
I20260709 12:14:06.413535 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:06.103005 26383 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.411289 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:06.433382 23853 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:06.361824 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.289625 26306 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:06.290761 26491 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:06.439200 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:06.441476 23854 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:06.349234 26502 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:06.334815 26370 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:06.477649 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 482304us
W20260709 12:14:06.484192 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.433629 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.430536 26373 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:06.434404 26362 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:14:06.475368 26508 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:06.476979 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:06.532562 26406 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:06.294423 25170 rpcz_store.cc:276] 0709 12:14:01.878426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.883511 (+  5085us) service_pool.cc:224] Handling call
0709 12:14:02.453905 (+570394us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:02.454137 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.164183 (+2710046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.703441 (+539258us) spinlock_profiling.cc:243] Waited 2.6 s on lock 0x7b54002526d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:05.703677 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5987408512}
I20260709 12:14:06.535398 26512 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:06.539515 26508 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:06.561066 26383 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:06.581058 26592 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:06.581595 26592 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:06.582501 26592 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.584111 26503 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:06.586198 26545 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:06.591058 26504 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.157s	user 0.007s	sys 0.000s
I20260709 12:14:06.600615 26443 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:06.602690 26451 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.272s	user 0.010s	sys 0.000s
I20260709 12:14:06.593981 26592 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:06.563740 26491 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.570997 26476 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:06.603349 26508 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:06.612444 26508 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:06.615231 26476 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:06.675948 23749 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.000s	sys 0.020s
W20260709 12:14:06.676806 23749 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.001s	sys 0.020s
W20260709 12:14:06.677352 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 702108us
I20260709 12:14:06.292347 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.680366 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.389060 23779 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:06.682286 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 281) took 1003 ms. Trace:
I20260709 12:14:06.682408 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2302) took 1276 ms. Trace:
I20260709 12:14:06.680312 23777 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:06.683182 23781 rpcz_store.cc:276] 0709 12:14:05.405607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.379776 (+974169us) service_pool.cc:224] Handling call
0709 12:14:06.682176 (+302400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.422983 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2294) took 1031 ms. Trace:
I20260709 12:14:06.686416 23782 rpcz_store.cc:276] 0709 12:14:05.402373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.396852 (+994479us) service_pool.cc:224] Handling call
0709 12:14:06.397156 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.422679 (+ 25523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.422918 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.686710 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:06.682783 23779 rpcz_store.cc:276] 0709 12:14:05.678861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.291196 (+612335us) service_pool.cc:224] Handling call
0709 12:14:06.682066 (+390870us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.688145 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:06.689502 23779 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:14:06.690388 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2238) took 1283 ms. Trace:
I20260709 12:14:06.690536 23780 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:06.680980 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2243) took 1272 ms. Trace:
I20260709 12:14:06.691334 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2240) took 1283 ms. Trace:
I20260709 12:14:06.691875 23780 rpcz_store.cc:276] 0709 12:14:05.407634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.689568 (+1281934us) service_pool.cc:224] Handling call
0709 12:14:06.690141 (+   573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.691063 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.691255 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.681222 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2299) took 1276 ms. Trace:
I20260709 12:14:06.692760 23778 rpcz_store.cc:276] 0709 12:14:05.404846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.254644 (+849798us) service_pool.cc:224] Handling call
0709 12:14:06.680990 (+426346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.694064 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2241) took 1286 ms. Trace:
I20260709 12:14:06.694680 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2242) took 1286 ms. Trace:
I20260709 12:14:06.695107 23778 rpcz_store.cc:276] 0709 12:14:05.408120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.693556 (+1285436us) service_pool.cc:224] Handling call
0709 12:14:06.694008 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.694339 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.694364 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.694419 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:06.694479 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.694540 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.694630 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.695986 23784 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:06.691489 23776 rpcz_store.cc:276] 0709 12:14:05.408376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.441582 (+ 33206us) service_pool.cc:224] Handling call
0709 12:14:06.680366 (+1238784us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400167454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.680658 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815320960}
I20260709 12:14:06.699334 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2237) took 1292 ms. Trace:
I20260709 12:14:06.699903 23784 rpcz_store.cc:276] 0709 12:14:05.406863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.441445 (+ 34582us) service_pool.cc:224] Handling call
0709 12:14:06.698945 (+1257500us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540035a2d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.699229 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1677680000}
I20260709 12:14:06.700865 23776 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:06.701668 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 279) took 1023 ms. Trace:
I20260709 12:14:06.702025 23776 rpcz_store.cc:276] 0709 12:14:05.678203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.700043 (+1021840us) service_pool.cc:224] Handling call
0709 12:14:06.700552 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.701382 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.701600 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.702625 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 282) took 1023 ms. Trace:
I20260709 12:14:06.690779 23779 rpcz_store.cc:276] 0709 12:14:05.407116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.688427 (+1281311us) service_pool.cc:224] Handling call
0709 12:14:06.689035 (+   608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.690017 (+   982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.690276 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.704039 23782 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:14:06.691188 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:06.706200 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2301) took 1300 ms. Trace:
I20260709 12:14:06.706712 23782 rpcz_store.cc:276] 0709 12:14:05.405365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.687754 (+1282389us) service_pool.cc:224] Handling call
0709 12:14:06.703708 (+ 15954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.704537 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.705847 (+  1310us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b540035a2d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.706081 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35388160}
I20260709 12:14:06.707921 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:06.708997 23782 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:14:06.709129 23785 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:06.713292 23785 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:06.718849 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 275) took 8866 ms. Trace:
W20260709 12:14:06.719838 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.008s	sys 0.015s
W20260709 12:14:06.720338 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.009s	sys 0.015s
W20260709 12:14:06.720893 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 752420us
I20260709 12:14:06.722085 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:06.703297 23784 rpcz_store.cc:276] 0709 12:14:05.679123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.701073 (+1021950us) service_pool.cc:224] Handling call
0709 12:14:06.701736 (+   663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.702192 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.702221 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.702299 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:06.702363 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.702423 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.702553 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.726037 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:06.726752 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:06.727224 23851 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:06.707237 23749 leader_election.cc:400] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:06.729360 23749 leader_election.cc:403] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:14:06.729219 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:06.706884 23779 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:06.727491 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.737668 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.033s	sys 0.033s
W20260709 12:14:06.738214 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.033s	sys 0.033s
W20260709 12:14:06.738611 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.738706 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 722628us
I20260709 12:14:06.736896 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 283) took 1057 ms. Trace:
I20260709 12:14:06.740061 23779 rpcz_store.cc:276] 0709 12:14:05.679404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.703723 (+1024319us) service_pool.cc:224] Handling call
0709 12:14:06.704207 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.736621 (+ 32414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.736808 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.742700 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:06.734535 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:06.743628 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.010s	sys 0.020s
W20260709 12:14:06.743032 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.016s	sys 0.049s
W20260709 12:14:06.744071 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.010s	sys 0.021s
W20260709 12:14:06.744436 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.016s	sys 0.049s
W20260709 12:14:06.744999 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.744242 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:06.745688 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.738509 23856 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:06.732111 23782 pending_rounds.cc:85] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
W20260709 12:14:06.745705 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.694525 23780 rpcz_store.cc:276] 0709 12:14:05.407875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.692877 (+1285002us) service_pool.cc:224] Handling call
0709 12:14:06.693307 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.693709 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.693732 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.693792 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:06.693821 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.693882 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.694005 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.746294 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 767186us
I20260709 12:14:06.743969 23852 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:06.761931 23777 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:06.734596 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:06.768616 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.747684 23782 pending_rounds.cc:107] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:14:06.770210 23782 raft_consensus.cc:2889] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:06.772058 26502 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:06.773285 26502 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:06.774248 26502 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:06.781813 23781 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:06.783054 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 280) took 1104 ms. Trace:
I20260709 12:14:06.783391 23777 rpcz_store.cc:276] 0709 12:14:05.678486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.292100 (+613614us) service_pool.cc:224] Handling call
0709 12:14:06.779318 (+487218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.789696 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 195) took 1074 ms. Trace:
I20260709 12:14:06.793277 23857 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:06.793965 26587 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:06.795079 25171 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:06.795745 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.014s	sys 0.036s
W20260709 12:14:06.796250 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.014s	sys 0.036s
W20260709 12:14:06.796675 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 800754us
I20260709 12:14:06.565621 26306 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.802847 26376 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:06.803428 26376 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:06.804052 26376 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:06.804261 26591 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:06.793341 26364 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:06.809305 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:06.809713 26512 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:06.810453 26512 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.811008 26512 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:06.795020 26587 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:06.812330 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:06.813817 25172 raft_consensus.cc:2410] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:06.814440 26587 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:06.815258 23860 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:06.815647 23932 raft_consensus.cc:2410] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:06.817449 26393 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:06.817991 26393 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:06.809322 23859 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:06.824283 26503 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:06.824877 26503 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:06.825309 26503 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:06.827817 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83777.3us
I20260709 12:14:06.833748 26383 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:06.834344 26383 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.834746 26383 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:06.567888 26373 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.809715 26389 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.820184 26393 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:06.855304 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.856135 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:06.856616 26504 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.857501 26504 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:06.860713 23751 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.666s (SENT)
W20260709 12:14:06.862241 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.862422 26405 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:14:06.791370 25173 rpcz_store.cc:276] 0709 12:14:05.714719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.432608 (+717889us) service_pool.cc:224] Handling call
0709 12:14:06.789420 (+356812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.805126 25177 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:06.876343 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2296) took 1472 ms. Trace:
I20260709 12:14:06.876857 23781 rpcz_store.cc:276] 0709 12:14:05.404095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.685320 (+1281225us) service_pool.cc:224] Handling call
0709 12:14:06.763125 (+ 77805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.790438 (+ 27313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.876036 (+ 85598us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b540010bb50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.876250 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175946496}
I20260709 12:14:06.883919 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 228) took 1611 ms. Trace:
I20260709 12:14:06.884410 23928 rpcz_store.cc:276] 0709 12:14:05.178701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.059101 (+880400us) service_pool.cc:224] Handling call
0709 12:14:06.068691 (+  9590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.441214 (+372523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.441258 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.441734 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.764762 (+323028us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:06.764868 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.765011 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.767645 (+  2634us) raft_consensus.cc:1685] finished
0709 12:14:06.767662 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.790464 (+ 22802us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67760,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":694419,"threads_started":2,"wal-append.queue_time_us":322740}
I20260709 12:14:06.886418 26389 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:06.887841 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.887989 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:06.888352 26504 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:06.810384 26443 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:14:06.889001 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 197) took 1137 ms. Trace:
I20260709 12:14:06.889501 25176 rpcz_store.cc:276] 0709 12:14:05.752127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.887479 (+1135352us) service_pool.cc:224] Handling call
0709 12:14:06.888914 (+  1435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.863437 26508 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:06.890982 26508 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:06.891464 26508 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:06.895915 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 83890.6us
I20260709 12:14:06.896939 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:06.898599 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:06.901180 26493 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:06.903717 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:06.903963 26484 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:06.905211 26484 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:06.905663 26484 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:06.907670 26568 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:06.909485 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:06.909821 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:06.910145 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:06.910419 23927 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:06.910792 23931 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:06.912581 26415 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:06.913166 26463 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:06.915135 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.916031 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:06.917234 23928 raft_consensus.cc:2410] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:06.905577 26610 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:06.920408 26430 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:06.921172 26430 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:06.921546 26512 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:06.905251 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 63324.9us
I20260709 12:14:06.922276 26430 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:06.923288 26259 consensus_meta.cc:287] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.499s	user 0.004s	sys 0.008s
I20260709 12:14:06.913201 23854 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:06.923889 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:06.924993 25172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 443) took 1862 ms (client timeout 1898 ms). Trace:
I20260709 12:14:06.936190 26599 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:06.911052 23934 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:06.882058 25174 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:06.936631 26383 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:06.896807 25169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 646) took 3836 ms. Trace:
I20260709 12:14:06.905354 26586 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:07.003700 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:07.003911 23751 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.758s (SENT)
I20260709 12:14:06.906925 26484 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:06.966470 25172 rpcz_store.cc:269] 0709 12:14:05.062088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.824176 (+762088us) service_pool.cc:224] Handling call
0709 12:14:06.924708 (+1100532us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400492854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.924910 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764152704}
I20260709 12:14:06.719283 23683 rpcz_store.cc:276] 0709 12:13:57.852066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.852260 (+   194us) service_pool.cc:224] Handling call
0709 12:14:06.718576 (+8866316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:02.459104 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:02.682218 (+223114us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:02.686381 (+  4163us) write_op.cc:435] Acquired schema lock
0709 12:14:02.688796 (+  2415us) tablet.cc:662] Decoding operations
0709 12:14:02.820918 (+132122us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:02.845457 (+ 24539us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:02.848859 (+  3402us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:14:02.928758 (+ 79899us) tablet.cc:701] Row locks acquired
0709 12:14:02.928789 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:14:02.928940 (+   151us) write_op.cc:270] Start()
0709 12:14:02.958443 (+ 29503us) write_op.cc:276] Timestamp: P: 1783599242928883 usec, L: 0
0709 12:14:02.979765 (+ 21322us) op_driver.cc:348] REPLICATION: starting
0709 12:14:03.048570 (+ 68805us) log.cc:844] Serialized 3984 byte log entry
0709 12:14:05.538187 (+2489617us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:06.585526 (+1047339us) op_driver.cc:464] REPLICATION: finished
0709 12:14:06.616110 (+ 30584us) write_op.cc:301] APPLY: starting
0709 12:14:06.616189 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:06.616393 (+   204us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:06.616413 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:06.617587 (+  1174us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:06.617822 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:14:06.617872 (+    50us) write_op.cc:312] APPLY: finished
0709 12:14:06.618129 (+   257us) log.cc:844] Serialized 77 byte log entry
0709 12:14:06.619021 (+   892us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:06.619164 (+   143us) write_op.cc:454] Released schema lock
0709 12:14:06.619212 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:06.619630 (+   418us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:06.619657 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":747081,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":30321,"apply.run_cpu_time_us":2751,"apply.run_wall_time_us":3469,"num_ops":10,"prepare.queue_time_us":265433,"prepare.run_cpu_time_us":92963,"prepare.run_wall_time_us":831918,"raft.queue_time_us":454799,"raft.run_cpu_time_us":7699,"raft.run_wall_time_us":33392,"replication_time_us":3605582,"spinlock_wait_cycles":11276160,"thread_start_us":886393,"threads_started":4,"wal-append.queue_time_us":332698}]]}
I20260709 12:14:06.978264 26443 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:07.011741 26586 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:06.971451 26259 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.018301 26398 raft_consensus.cc:2764] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:07.048048 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.050062 26306 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:07.044399 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 287779us
I20260709 12:14:07.030239 26484 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:07.063764 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 140039us
I20260709 12:14:07.061055 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:07.080255 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:07.081224 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 200) took 1134 ms. Trace:
I20260709 12:14:07.081542 25175 rpcz_store.cc:276] 0709 12:14:05.946476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.056072 (+1109596us) service_pool.cc:224] Handling call
0709 12:14:07.081141 (+ 25069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:07.030460 26599 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.075799 26306 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.084007 23783 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:07.067798 26592 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:14:07.095337 26586 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:07.096695 26370 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:14:07.111912 26370 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.113351 26306 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.114451 26370 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.097671 23680 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:07.088338 26602 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:07.060120 26512 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.109395 23929 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:07.118449 26591 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.119146 26383 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.119096 26591 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.120270 25114 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:07.121208 26613 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:14:07.129935 26376 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.133414 26586 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:07.134963 23783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2233) took 1729 ms (client timeout 2205 ms). Trace:
W20260709 12:14:07.135406 23783 rpcz_store.cc:269] 0709 12:14:05.405844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.937866 (+532022us) service_pool.cc:224] Handling call
0709 12:14:07.134720 (+1196854us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540041c654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:07.134895 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2511677184}
I20260709 12:14:07.136567 23778 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:07.137545 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 78373.3us
I20260709 12:14:07.138022 26393 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:07.138815 25119 leader_election.cc:400] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:07.139165 25119 leader_election.cc:403] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:14:07.141711 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 66188.9us
I20260709 12:14:07.142628 25967 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:07.005671 25169 rpcz_store.cc:276] 0709 12:14:03.059970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.243640 (+183670us) service_pool.cc:224] Handling call
0709 12:14:05.638755 (+2395115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.294327 (+655572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.896355 (+602028us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b5400614250. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:06.896597 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5365147904}
W20260709 12:14:07.156272 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98798.4us
W20260709 12:14:07.162533 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.157990 26490 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.063828 26606 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:14:07.167903 26599 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.172478 26597 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:07.184159 26597 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:07.172648 26602 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:07.185264 26602 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:07.174227 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.133380 26376 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.118111 26503 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.152873 26587 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:07.196139 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 76035.1us
W20260709 12:14:07.202791 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.208287 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.210700 26591 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:07.212852 26503 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.214097 26508 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.215744 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 905) took 1887 ms. Trace:
I20260709 12:14:07.216343 26508 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:07.204118 26389 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:07.217320 23932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 896) took 1891 ms (client timeout 2233 ms). Trace:
W20260709 12:14:07.217716 23932 rpcz_store.cc:269] 0709 12:14:05.314387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.716527 (+402140us) service_pool.cc:224] Handling call
0709 12:14:07.199654 (+1483127us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54002a4254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:07.205353 (+  5699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1838359936}
I20260709 12:14:07.205160 26490 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.220912 26490 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:07.221441 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.223623 26259 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.224330 26398 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.224885 26398 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.226718 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:07.217242 25967 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:07.198645 26587 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:07.234266 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.251863 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:07.253439 23929 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:07.240491 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:07.240862 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.243103 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.241242 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.243546 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.240099 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.286274 23859 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:07.216122 23933 rpcz_store.cc:276] 0709 12:14:05.317347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.089584 (+772237us) service_pool.cc:224] Handling call
0709 12:14:06.089884 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.433210 (+343326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.433252 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.475761 (+ 42509us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.924646 (+448885us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:06.924699 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.924810 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.201771 (+276961us) raft_consensus.cc:1685] finished
0709 12:14:07.201788 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.204408 (+  2620us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46824,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":380582,"threads_started":2,"wal-append.queue_time_us":189121}
W20260709 12:14:07.289485 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.290409 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:07.237030 26373 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.297721 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 428) took 3861 ms. Trace:
I20260709 12:14:07.298453 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.243141 26491 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:07.298580 23932 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:07.281440 26568 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:07.185401 26597 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.272011 25168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 653) took 1714 ms. Trace:
I20260709 12:14:07.307448 26590 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.308120 23853 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:07.251046 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:07.297470 26306 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:07.320537 26531 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:07.321950 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 278) took 1663 ms. Trace:
I20260709 12:14:07.322114 26407 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.347174 23778 rpcz_store.cc:276] 0709 12:14:05.658854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.696834 (+1037980us) service_pool.cc:224] Handling call
0709 12:14:07.136363 (+439529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.136972 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.321721 (+184749us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540041c650. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:07.321871 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009753984}
I20260709 12:14:07.326695 26383 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:07.329964 26587 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.355201 26587 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:07.355614 26590 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.357941 26590 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.299084 25177 rpcz_store.cc:276] 0709 12:14:03.432472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.803570 (+3371098us) service_pool.cc:224] Handling call
0709 12:14:06.804214 (+   644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.804742 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:07.074065 (+269323us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.288316 (+214251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.288366 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:07.293399 (+  5033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.293472 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.293595 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2280,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4223,"thread_start_us":136206,"threads_started":1}
I20260709 12:14:07.350725 25169 raft_consensus.cc:606] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:07.352291 26531 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.358584 26572 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
W20260709 12:14:07.359835 26617 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.269s	user 0.004s	sys 0.001s
I20260709 12:14:07.361375 26568 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.353050 26407 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.351110 26383 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:07.332530 26520 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.344369 26490 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:07.373108 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 146059us
I20260709 12:14:07.306157 26373 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.377238 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.350251 26602 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.380582 23860 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:07.383885 26586 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:07.384990 26531 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:07.395569 25114 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:07.395391 26383 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.404124 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.404542 26586 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:07.405248 23860 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:07.405243 26586 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:07.407431 26586 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.407997 25967 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:07.409240 23858 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:07.385469 26484 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:14:07.412897 26383 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:07.417966 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.347995 26491 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.389128 26592 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:14:07.386919 26568 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.379833 26590 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:07.397620 26621 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 12:14:07.374571 26490 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.397117 26398 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.379446 26520 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:07.514674 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.518824 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:07.524304 25119 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:07.528304 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:07.512283 26407 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.527099 26370 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.539258 26370 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.540205 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:07.539511 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:07.542833 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.512253 26590 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:07.619030 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.588886 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.544173 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:07.647300 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 147846us
I20260709 12:14:07.654721 23932 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:07.675802 23853 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:07.747066 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.528668 26602 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:07.746538 23856 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
W20260709 12:14:07.748070 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 119211us
I20260709 12:14:07.821548 26520 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:07.821983 26370 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.822597 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:07.537701 26586 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:07.823820 26586 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:07.824791 26586 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:07.826088 26586 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:07.828754 26491 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:07.832423 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.833129 23856 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
W20260709 12:14:07.834901 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.844105 23859 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:07.854172 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:07.857371 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:07.859901 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.628873 26626 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:07.875293 23779 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:07.876431 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.878005 23782 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:07.884718 23932 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:07.885442 23934 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:07.886042 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:07.886888 25169 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
W20260709 12:14:07.887336 26619 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.537s	user 0.000s	sys 0.001s
I20260709 12:14:07.887925 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:07.889518 26590 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:07.870432 23852 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:07.343083 25168 rpcz_store.cc:276] 0709 12:14:05.355369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.760732 (+1405363us) service_pool.cc:224] Handling call
0709 12:14:06.817220 (+ 56488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.817825 (+   605us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:06.848106 (+ 30281us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.021544 (+173438us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.028780 (+  7236us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:07.029611 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.052734 (+ 23123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.070103 (+ 17369us) inbound_call.cc:177] Queueing success response
0709 12:14:07.133533 (+ 63430us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b18002e5fa0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2b168658f 00007fb2aa6c5571 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd
Metrics: {"mutex_wait_us":18856,"prepare.queue_time_us":188619,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1234,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":34734,"spinlock_wait_cycles":10679936,"thread_start_us":180028,"threads_started":2,"wal-append.queue_time_us":204908}
I20260709 12:14:07.878407 23782 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:07.893277 23782 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:07.909461 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.913266 23780 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:14:07.914633 23856 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:07.905733 23898 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:07.872535 26626 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.842919 26407 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:07.928000 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:07.941701 23782 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:07.923138 26626 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:07.978786 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.987212 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 81716.1us
I20260709 12:14:07.990269 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.000789 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:08.008565 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.946043 26590 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.051409 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.052423 23851 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:08.042693 26407 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.066999 26407 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:08.068624 26587 consensus_meta.cc:287] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.001s
I20260709 12:14:08.071281 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:08.076773 25174 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.041s	sys 0.027s
W20260709 12:14:08.077153 25174 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.041s	sys 0.027s
I20260709 12:14:08.079746 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.083877 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.088199 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.088654 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.089262 23853 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:08.089308 23859 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:08.090024 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.088164 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 430) took 4628 ms. Trace:
I20260709 12:14:08.033208 26491 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:14:08.096033 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.100090 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:08.115470 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 278841us
I20260709 12:14:08.053305 26484 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:14:08.116519 23781 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:14:08.124907 23899 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.126s (SENT)
I20260709 12:14:08.084904 26259 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:08.133541 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.132225 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.135259 23853 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:08.135838 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.092125 25174 rpcz_store.cc:276] 0709 12:14:03.449523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.880830 (+3431307us) service_pool.cc:224] Handling call
0709 12:14:06.881290 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.881774 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:07.152256 (+270482us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.077431 (+925175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.077481 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:08.077520 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.077578 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.077693 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4867,"raft.queue_time_us":78261,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"thread_start_us":866453,"threads_started":2,"wal-append.queue_time_us":380015}
I20260709 12:14:08.134187 23855 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:08.134970 23851 raft_consensus.cc:2376] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7. Re-sending same reply.
I20260709 12:14:08.149377 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:08.138047 26383 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.725s	user 0.009s	sys 0.004s
I20260709 12:14:08.151299 26259 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:08.167781 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.170970 23931 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
W20260709 12:14:08.176028 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 128043us
I20260709 12:14:08.178001 26259 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:08.179279 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2499) took 1299 ms. Trace:
I20260709 12:14:08.175468 23856 raft_consensus.cc:2410] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:14:08.179709 23857 rpcz_store.cc:276] 0709 12:14:06.880324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.882283 (+  1959us) service_pool.cc:224] Handling call
0709 12:14:06.882709 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.030429 (+147720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.044908 (+ 14479us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.116670 (+ 71762us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.340733 (+224063us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:07.340779 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.340901 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.177691 (+836790us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b540030a0a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad727dbe 00007fb2ad732930 00007fb2ad7328d8 00007fb2ad732858 00007fb2ad732813 00007fb2ad731a9c 00007fb2b361d2f6 00007fb2b3617aa8 00007fb2ad76e7cb 00007fb2ad7712c0
0709 12:14:08.178928 (+  1237us) raft_consensus.cc:1685] finished
0709 12:14:08.178949 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.179164 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179852,"raft.queue_time_us":146318,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":29535,"spinlock_wait_cycles":65661696,"thread_start_us":130633,"threads_started":1,"wal-append.queue_time_us":149594}
I20260709 12:14:08.168931 23785 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:08.183184 25119 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.184324 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.171072 23928 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:14:08.171830 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.172746 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.173727 23927 raft_consensus.cc:1240] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:08.174232 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:08.175203 26636 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:14:08.148746 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:08.193022 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 197326us
W20260709 12:14:08.189374 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57167.1us
I20260709 12:14:08.191254 23925 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:08.191375 26631 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:08.191905 26484 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:08.192364 26641 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:08.203526 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.202147 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.204015 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.196733 23852 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:08.196260 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.207926 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.206276 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.198556 23782 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:14:08.199080 23785 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:08.214676 23782 pending_rounds.cc:85] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 7
I20260709 12:14:08.220012 23784 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:08.220744 23782 pending_rounds.cc:107] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:14:08.221169 23782 raft_consensus.cc:2889] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:08.197703 26631 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:08.199386 26484 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.223953 23851 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:08.232255 23780 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:08.262265 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.263104 23853 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:08.269248 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:08.270186 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.274211 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.273732 23929 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:14:08.280308 23925 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:08.280388 26640 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.281028 26640 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.199141 26491 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:14:08.276412 23929 pending_rounds.cc:85] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 3
I20260709 12:14:08.280448 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.284168 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:08.263871 26631 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.205893 26590 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.288573 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.266845 26484 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.285609 23929 pending_rounds.cc:107] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:08.295125 23929 raft_consensus.cc:2889] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:08.293433 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.296690 23785 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
W20260709 12:14:08.297047 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 172195us
I20260709 12:14:08.269503 26634 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:14:08.297439 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 110235us
I20260709 12:14:08.299176 23784 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.302737 23782 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.304831 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.307026 23776 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:08.308884 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.324092 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:08.317534 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.326442 23784 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.327827 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.283551 26640 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:08.330179 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.331455 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.336076 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:08.337437 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.345814 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.347018 23925 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:08.329152 26586 consensus_meta.cc:287] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.000s
I20260709 12:14:08.357534 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.348582 23785 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.350817 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.353578 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.328760 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.349287 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.360394 23783 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.311326 26259 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:14:08.361054 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:08.355209 23824 connection.cc:256] RPC call timeout handler was delayed by -2.4482958979999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 12:14:08.364331 23824 connection.cc:256] RPC call timeout handler was delayed by -1.6869887740001559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:14:08.364697 23824 connection.cc:256] RPC call timeout handler was delayed by -1.614088774000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 12:14:08.365005 23824 connection.cc:256] RPC call timeout handler was delayed by -1.6077887740000278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 12:14:08.361953 23855 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
W20260709 12:14:08.365286 23824 connection.cc:256] RPC call timeout handler was delayed by -1.6005887740000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 12:14:08.365659 23824 connection.cc:256] RPC call timeout handler was delayed by -1.532188774000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:14:08.365965 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3836887740001202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:14:08.366235 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3674887740000941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 12:14:08.366501 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3370920800000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 12:14:08.366770 23824 connection.cc:256] RPC call timeout handler was delayed by -1.2801887740001803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
W20260709 12:14:08.367035 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1343887740001719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
W20260709 12:14:08.367293 23824 connection.cc:256] RPC call timeout handler was delayed by -1.055498488000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 12:14:08.356886 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.369410 23781 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.371408 23782 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.368124 23778 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:08.374495 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.375378 23785 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.376420 23931 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.382236 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.382511 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.383026 23783 raft_consensus.cc:2376] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8. Re-sending same reply.
I20260709 12:14:08.386072 23783 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:08.387689 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.389067 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:08.387455 23783 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:08.389616 23926 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.392035 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.396248 23930 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:08.397454 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.398273 23930 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.399124 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:08.370791 23776 raft_consensus.cc:2376] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6. Re-sending same reply.
I20260709 12:14:08.413453 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.420691 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:08.420879 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52765.1us
I20260709 12:14:08.421461 23785 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.427663 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.429073 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.432708 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.423771 26587 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:08.441941 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.444059 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.445952 23778 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:08.446120 23930 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:08.449354 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 142819us
W20260709 12:14:08.452174 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 69717.6us
I20260709 12:14:08.455695 23776 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:08.451843 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59014.8us
I20260709 12:14:08.457967 23783 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:08.459574 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:08.462216 23930 raft_consensus.cc:2410] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
W20260709 12:14:08.464330 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.469923 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:08.470810 25113 leader_election.cc:400] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5.
I20260709 12:14:08.471665 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.472317 26633 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:14:08.473905 26633 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:08.475900 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.471482 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.476667 25177 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:08.477080 23930 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:08.477679 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.478148 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.478838 23930 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:08.482142 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.483736 23930 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:08.475960 23855 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:14:08.463399 26644 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:14:08.495584 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:08.486963 26626 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.462960 26586 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.462791 26640 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:08.505303 26383 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.502019 23926 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:08.466291 23778 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 5 to 7, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:08.508224 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.476019 26645 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:08.512298 26645 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.514622 26645 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:08.517143 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.518535 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.518970 26651 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.519899 26259 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:14:08.520959 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.520877 26259 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:08.521520 26259 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:08.521890 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.522568 25175 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:08.522418 26259 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.523758 26259 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:08.525122 26590 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:08.453363 26649 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:14:08.527619 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61640.8us
I20260709 12:14:08.528193 26649 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:08.528478 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.528676 26649 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:08.458307 26648 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:14:08.529738 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.529594 26649 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:08.531067 26649 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:08.532444 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.533861 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 66616.4us
I20260709 12:14:08.536933 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.549943 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.545956 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.592948 25168 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:08.593140 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.543983 26520 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.594455 26259 raft_consensus.cc:2764] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:08.595284 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.551108 23780 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:08.551867 26631 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.596858 26259 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.597132 23928 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:08.596115 23785 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:08.519773 26651 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.554359 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.535408 26491 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.488662 26639 raft_consensus.cc:993] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:08.601280 26383 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:08.559708 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.602296 25174 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:08.560412 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.560672 25172 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:08.601689 26383 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:08.561483 26626 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:08.605183 26383 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.561604 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:08.562196 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.562363 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.562981 26654 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:08.564785 26490 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:08.570883 26430 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.574802 23931 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:08.575163 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.575954 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.576385 23855 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:08.577708 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 296) took 1040 ms. Trace:
I20260709 12:14:08.578630 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2492) took 3050 ms. Trace:
I20260709 12:14:08.581470 26597 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.609308 23777 rpcz_store.cc:276] 0709 12:14:07.537045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.541079 (+  4034us) service_pool.cc:224] Handling call
0709 12:14:07.591916 (+ 50837us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:07.654197 (+ 62281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.654238 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:07.868396 (+214158us) log.cc:844] Serialized 52 byte log entry
0709 12:14:08.140610 (+272214us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:08.415689 (+275079us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.465707 (+ 50018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.465786 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.577495 (+111709us) raft_consensus.cc:1685] finished
0709 12:14:08.577516 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.577648 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15252,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":38187,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":218330,"thread_start_us":511305,"threads_started":3,"wal-append.queue_time_us":371268}
I20260709 12:14:08.588114 26407 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:08.609395 23858 rpcz_store.cc:276] 0709 12:14:05.528029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.528161 (+   132us) service_pool.cc:224] Handling call
0709 12:14:05.528467 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.409597 (+1881130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.578380 (+1168783us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54002d3cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:08.578556 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4300992384}
I20260709 12:14:08.599155 23925 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:08.581269 26484 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.605005 23781 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:08.585405 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.561973 26645 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.617255 23784 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:08.617643 23934 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:14:08.619143 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 120012us
I20260709 12:14:08.620971 23930 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:08.622695 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.626355 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.627694 23785 raft_consensus.cc:2410] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:08.620795 26415 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:08.628769 23784 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:08.629057 26415 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:08.630038 26415 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.631299 26415 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.639940 26640 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:08.640964 26639 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:08.653013 26639 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:08.653492 26631 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:08.653962 26639 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.655205 26639 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.641413 26430 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.615389 26407 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:08.657680 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.656759 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.658906 23858 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:08.662039 23680 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.428s	user 0.052s	sys 0.035s
W20260709 12:14:08.644398 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.644935 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 113378us
I20260709 12:14:08.666543 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.668483 26590 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:08.669152 26590 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:08.669782 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.670754 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:14:08.671682 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.673229 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.674748 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.675313 23777 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:08.678260 23858 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:08.678960 26491 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:08.679380 26491 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:08.680260 26491 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:08.681144 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.681404 26491 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:08.681658 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.683048 26645 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:08.683665 26645 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.623780 26490 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:08.684212 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.684101 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.684898 23859 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:08.684798 26490 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:08.687096 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.688076 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.688697 23859 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:08.689664 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.691763 23933 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:08.691934 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.646047 23925 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:08.651944 26662 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.693707 26662 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.687114 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.695011 23860 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
W20260709 12:14:08.696272 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.696686 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.658459 26407 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.692656 23851 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:14:08.643396 26634 raft_consensus.cc:1081] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:08.687994 23929 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:08.707644 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 157779us
I20260709 12:14:08.700155 23777 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:08.701105 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.702839 23853 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:08.705570 25175 raft_consensus.cc:2368] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
I20260709 12:14:08.710439 23784 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:14:08.710680 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.710906 23784 pending_rounds.cc:85] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:14:08.711305 23784 pending_rounds.cc:107] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:14:08.713075 23784 raft_consensus.cc:2889] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:08.712168 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.711324 23855 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:14:08.714058 23680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.480s	user 0.054s	sys 0.035s
I20260709 12:14:08.692332 26430 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:08.719456 25119 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:08.728034 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.734642 23899 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.619s (SENT)
I20260709 12:14:08.735028 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.738601 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.707314 26590 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:14:08.743621 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.762162 23778 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:08.763558 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44178.9us
W20260709 12:14:08.766695 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54598.3us
I20260709 12:14:08.732713 26430 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.768324 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:08.768257 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.736380 26654 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:08.769660 23784 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:08.769140 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.736646 26626 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:08.726575 26586 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:14:08.773691 26669 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:08.765764 26665 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.770301 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.780788 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.777848 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.783046 23898 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.787330 23778 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 3.
I20260709 12:14:08.772356 26587 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.792817 25169 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:08.800413 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 72383.2us
W20260709 12:14:08.801882 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67260.6us
I20260709 12:14:08.776279 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:08.802834 23777 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:08.803133 23777 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:08.775326 26667 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:08.803750 26667 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.804159 26667 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.800254 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.771317 26430 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:08.814302 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.816437 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.799865 26592 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:08.819617 26592 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.795543 26669 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.820013 26592 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.821255 26383 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.821806 26669 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:08.778834 26654 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.824296 26654 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:08.813508 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.793833 26626 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:08.825945 26656 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:14:08.826192 26626 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.829481 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.831030 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.831303 26673 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:08.830086 23784 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:08.829710 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.803170 26648 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:14:08.836194 26664 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:08.839556 26306 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:08.840094 26306 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.844870 23781 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:08.834144 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.795s (SENT)
I20260709 12:14:08.846858 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:08.862210 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.872082 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.872489 26670 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:08.873005 26670 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:08.873404 26670 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.844439 26586 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.874416 26586 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.815642 26490 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:08.876767 26662 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:08.877581 26645 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:08.878006 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61881.3us
I20260709 12:14:08.879554 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.881788 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.882957 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.846359 26673 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.794232 26665 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:08.880259 23782 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:08.888453 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:08.908427 23776 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:08.886785 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.909642 26658 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.910420 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.911335 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.911902 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.912063 25169 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:08.887243 23929 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:08.882563 23779 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:08.913825 26259 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:08.913874 26658 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:08.914479 26658 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.914861 26658 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.915093 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:08.915014 26670 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.916133 26259 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:08.916643 26259 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:08.917392 26670 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:08.918634 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.919478 26649 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:08.920840 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:08.917644 26259 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.925366 26649 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:08.927181 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:08.928318 26649 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:08.928697 26649 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:08.929057 26649 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.929437 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.931439 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.933482 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.936267 23853 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 12:14:08.939328 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 56436us
I20260709 12:14:08.939378 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:08.940701 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.945029 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:08.944024 23777 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:08.945160 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 144968us
I20260709 12:14:08.946939 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:08.889104 26671 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:08.907476 26520 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:08.950768 23780 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
I20260709 12:14:08.907435 26597 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.907245 26673 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:08.956894 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 159579us
I20260709 12:14:08.940088 26651 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.954216 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.954969 23777 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:08.954921 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 171939us
I20260709 12:14:08.961365 23898 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.962337 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.962828 25119 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.923261 26658 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:08.966136 23855 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:08.952692 26520 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.967460 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.969760 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.967546 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:08.969676 26640 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:08.971657 23853 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:08.972015 23851 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.973351 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:08.972843 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:08.975328 23859 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:08.976171 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:08.979758 23852 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:08.985114 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:08.972947 26520 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.983314 26490 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:09.023942 26490 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:09.024401 26490 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:09.025102 26592 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:08.962191 26597 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.029835 26597 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.029484 26306 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.036098 26520 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:08.991994 26657 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.041939 26657 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:09.042352 26657 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.993242 26640 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.044410 26640 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.049149 23783 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:09.050312 25967 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:09.050742 25967 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.052167 26430 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:08.980840 26491 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:09.053615 26491 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.053992 26491 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.998008 26602 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:08.998360 26649 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.055136 26602 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.055941 26592 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.056766 26649 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:09.010169 26658 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.013226 26665 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:08.974031 26675 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:14:09.061492 26373 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:09.156805 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.982581 26626 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.230s
I20260709 12:14:09.036706 26383 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:09.277572 26306 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:08.990176 26667 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.280297 26667 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:09.281735 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.282462 26639 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.625s	user 0.008s	sys 0.001s
I20260709 12:14:09.295433 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.314325 26586 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.054883 26657 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.345867 26673 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.354796 26651 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:09.355919 23776 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:09.361371 26673 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:09.364773 26657 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.367131 23783 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:09.375552 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.382040 23779 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:09.401330 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 433829us
I20260709 12:14:09.402727 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.405833 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:09.406055 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.407531 26651 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:09.412086 26673 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.412642 26651 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.413182 26651 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:09.412706 26673 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.413554 26651 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.412891 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.415789 26657 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:09.418079 26673 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.418571 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.419378 26657 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:09.419806 26657 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:09.420290 26657 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.420961 23851 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:09.422782 26651 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.429864 26651 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.419852 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.430826 26658 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:09.434664 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.435634 26649 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:09.436869 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.437458 26484 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.437649 26649 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:09.438524 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.447264 26649 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:09.442852 26259 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.016s	sys 0.026s
W20260709 12:14:09.448163 26259 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.017s	sys 0.027s
I20260709 12:14:09.449270 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.450608 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.450881 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.451141 26259 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:09.452123 26649 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:09.452082 23780 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:09.453783 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.443560 26657 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.454572 26649 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:09.455274 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.456102 26649 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.456483 26649 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.456809 26649 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.459520 26657 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:09.464303 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 479218us
I20260709 12:14:09.465981 26634 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:09.466475 26634 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.466837 26634 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.468473 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.470053 26259 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:09.470748 26259 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:09.471571 26651 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:09.472003 26651 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:09.472306 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.472929 26651 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.474308 26651 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.479028 26657 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.464587 26649 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.483171 26649 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.487344 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 925) took 1111 ms. Trace:
I20260709 12:14:09.488621 23926 rpcz_store.cc:276] 0709 12:14:08.375879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.507104 (+131225us) service_pool.cc:224] Handling call
0709 12:14:08.507454 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.565400 (+ 57946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.565430 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:08.565899 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:14:08.705594 (+139695us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:08.739558 (+ 33964us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.739775 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.486975 (+747200us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54002b7854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:09.487021 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.487065 (+    44us) raft_consensus.cc:1685] finished
0709 12:14:09.487157 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.487280 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13902,"prepare.queue_time_us":53979,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1121,"raft.queue_time_us":67404,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":19503,"spinlock_wait_cycles":130013440,"thread_start_us":139294,"threads_started":1,"wal-append.queue_time_us":197208}
I20260709 12:14:09.488112 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.490558 26655 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:14:09.491706 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.492163 26306 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:09.493297 26657 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:09.495015 26663 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:09.495460 26663 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.495698 26649 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.496203 26649 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.498414 26649 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.500365 23785 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:09.501969 23785 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:09.502897 26639 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.503683 23785 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:09.503456 26639 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.496441 26663 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.505219 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.505504 26663 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:09.505962 23785 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.505704 26639 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.507247 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.507987 23785 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.507862 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.508610 23776 raft_consensus.cc:2376] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9. Re-sending same reply.
I20260709 12:14:09.501829 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.509187 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.509666 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:09.513036 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.513599 23776 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:09.519848 23785 raft_consensus.cc:2410] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:09.520730 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.522716 26383 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:09.524493 23780 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:09.524796 26383 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:09.524039 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.526227 26383 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.527117 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.528903 25114 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:09.531924 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.532673 23784 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.533730 26639 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:09.534758 26662 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:09.535123 26662 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.535989 26662 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.537295 26662 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.526603 26383 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:09.538125 26383 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.523114 25169 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:09.531567 26634 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.540270 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.543547 23785 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.545509 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.546391 23780 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.548141 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:09.547626 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 45867.8us
I20260709 12:14:09.548795 23785 raft_consensus.cc:2368] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
W20260709 12:14:09.551230 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.554126 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.554592 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.554914 23780 raft_consensus.cc:2376] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9. Re-sending same reply.
I20260709 12:14:09.543503 26484 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:09.562167 26689 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:09.562858 26689 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:09.563905 26689 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.565248 26689 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:09.577186 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.578850 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.579643 23784 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:09.581614 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:09.582468 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.583017 23781 raft_consensus.cc:2376] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6. Re-sending same reply.
I20260709 12:14:09.584517 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.585345 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.586211 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.588066 23781 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:09.591526 26602 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.592622 26602 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:09.593714 26383 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.588851 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:09.596809 23781 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:09.598639 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.598613 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.600095 26649 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:09.599611 23784 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.608881 26373 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:09.609894 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.610265 26373 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:09.611336 23780 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.616370 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.599139 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.618053 26649 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:09.619870 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.620498 26634 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.622222 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.625939 23781 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.622932 26649 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:09.630052 26649 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:09.631093 26649 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:09.631979 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 75874.7us
I20260709 12:14:09.633526 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.634418 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.635134 26587 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:14:09.637300 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.015s	sys 0.033s
W20260709 12:14:09.637645 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.016s	sys 0.034s
I20260709 12:14:09.637879 26689 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:09.638306 26689 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.638633 26689 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.639818 25175 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:09.640856 25172 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
W20260709 12:14:09.641165 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 668334us
I20260709 12:14:09.642916 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.643764 23778 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.644702 26692 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:09.647408 26669 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:09.066180 26490 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:09.717203 26415 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.011s	sys 0.003s
I20260709 12:14:09.716354 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:09.716012 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:09.700574 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:09.727948 23857 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:09.716005 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:09.719570 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 758159us
I20260709 12:14:09.646585 26681 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.661484 26484 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.732517 23854 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.666689 25967 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.699028 23852 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:09.734730 26572 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:09.728904 26415 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.013s	sys 0.004s
I20260709 12:14:09.702695 26651 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.735316 26572 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.735850 26651 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.737563 26572 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.695055 26520 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:09.656637 23855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.009s	sys 0.016s
W20260709 12:14:09.738787 23855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.013s	sys 0.017s
I20260709 12:14:09.701035 26407 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.683490 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:09.666405 26491 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.005s
I20260709 12:14:09.741536 23933 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:09.681391 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:09.655511 26490 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:09.704820 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:09.742622 23925 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:09.743016 23860 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:14:09.673691 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.732141 26669 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:14:09.668417 26692 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:09.745740 26692 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.746394 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:09.746531 23927 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:09.733495 26484 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:09.678905 23859 consensus_meta.cc:287] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.000s
I20260709 12:14:09.710088 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:09.741801 26491 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.733072 23851 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:09.749788 23859 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:09.751451 26491 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:09.752290 23856 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.752614 25967 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.624209 25177 raft_consensus.cc:606] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:09.776680 25177 raft_consensus.cc:614] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:09.777235 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50586 (request call id 1) took 1032 ms. Trace:
W20260709 12:14:09.749125 23680 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 268) took 13062 ms (client timeout 15000 ms). Trace:
I20260709 12:14:09.777539 25177 rpcz_store.cc:276] 0709 12:14:08.744595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.744758 (+   163us) service_pool.cc:224] Handling call
0709 12:14:09.624063 (+879305us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278ec40 00007fb2b277abb9 00007fb2b277a47c 00007fb2abd007d4
0709 12:14:09.777126 (+153063us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532608}
I20260709 12:14:09.777788 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:09.730751 26681 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.782263 26681 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.732044 23898 leader_election.cc:400] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
I20260709 12:14:09.784924 23898 leader_election.cc:403] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:09.744586 26669 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:09.667310 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.671470 26697 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:14:09.764148 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:09.785691 26669 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:09.786186 23927 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:09.787010 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 119791us
I20260709 12:14:09.786804 26669 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.787695 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.789641 26697 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:09.778132 23680 rpcz_store.cc:269] 0709 12:13:56.686964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.687193 (+   229us) service_pool.cc:224] Handling call
0709 12:14:08.714472 (+12027279us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:08.714502 (+    30us) write_op.cc:454] Released schema lock
0709 12:14:09.749064 (+1034562us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:07.521306 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:07.522835 (+  1529us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:07.522888 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:14:07.522908 (+    20us) tablet.cc:662] Decoding operations
0709 12:14:07.523369 (+   461us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:07.523459 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:07.523486 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:14:07.523876 (+   390us) tablet.cc:701] Row locks acquired
0709 12:14:07.523923 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:14:07.524122 (+   199us) write_op.cc:270] Start()
0709 12:14:07.524283 (+   161us) write_op.cc:276] Timestamp: P: 1783599247524045 usec, L: 0
0709 12:14:07.524303 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:14:07.525104 (+   801us) log.cc:844] Serialized 533 byte log entry
0709 12:14:08.060142 (+535038us) op_driver.cc:464] REPLICATION: finished
0709 12:14:08.461839 (+401697us) write_op.cc:301] APPLY: starting
0709 12:14:08.489140 (+ 27301us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:08.501033 (+ 11893us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:08.502943 (+  1910us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:08.503249 (+   306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:08.503488 (+   239us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:14:08.503531 (+    43us) write_op.cc:312] APPLY: finished
0709 12:14:08.503830 (+   299us) log.cc:844] Serialized 23 byte log entry
0709 12:14:08.504060 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:08.504141 (+    81us) write_op.cc:454] Released schema lock
0709 12:14:08.504195 (+    54us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":369431,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":371285,"apply.run_cpu_time_us":2211,"apply.run_wall_time_us":52183,"num_ops":1,"prepare.queue_time_us":287578,"prepare.run_cpu_time_us":3492,"prepare.run_wall_time_us":20062,"replication_time_us":535631,"thread_start_us":1714272,"threads_started":2}]]}
I20260709 12:14:09.764443 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:09.804195 23928 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:09.767012 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.805065 23776 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:09.767169 23778 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:09.768291 23854 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:09.764710 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:09.774127 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 140880us
I20260709 12:14:09.622531 26383 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.780526 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:09.780774 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:09.791886 26373 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:09.793931 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.763644 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:09.795046 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:09.793100 23859 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.796519 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:09.774832 23681 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 4 to 6, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 6 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.797187 23927 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:09.815308 23934 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.738365 26651 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.748807 26689 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.623803 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.762902 23856 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:09.812711 23855 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.813186 25177 raft_consensus.cc:606] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:09.831071 25177 raft_consensus.cc:614] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:09.812198 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.831485 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:09.818581 23781 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:09.833779 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:09.804924 26697 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:09.835975 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:09.845729 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 113794us
W20260709 12:14:09.848287 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 224503us
W20260709 12:14:09.833736 23899 leader_election.cc:336] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.513s (SENT)
I20260709 12:14:09.857036 23899 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:09.815075 26692 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:14:09.860059 23928 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:09.841717 26520 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:09.877981 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.878851 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.879119 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.838757 26651 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:09.863817 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.815382 26590 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:14:09.843005 26572 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:09.869107 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:09.869328 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 72280.2us
I20260709 12:14:09.870620 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:09.862639 23825 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.036s (SENT)
I20260709 12:14:09.883863 23781 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.911167 25177 raft_consensus.cc:2368] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5.
I20260709 12:14:09.843608 26306 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:09.846815 26673 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:09.911307 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:09.913163 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44156.7us
I20260709 12:14:09.913321 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:09.913767 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:09.911521 23932 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:09.913373 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:09.922286 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.927233 23777 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:09.921325 23859 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:14:09.923574 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 53100.6us
I20260709 12:14:09.931198 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.932071 23780 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:09.923931 25176 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:09.925962 23932 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:09.936146 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.878494 26520 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.936859 23781 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.932027 23778 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:09.912762 26572 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:09.924244 26673 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.924031 26306 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.932646 26675 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:09.925796 26602 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:09.930392 26490 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:09.914592 26383 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:14:09.935708 23932 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.936343 26657 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:09.939715 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.962378 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:09.937354 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 101452us
I20260709 12:14:09.964848 23859 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:09.955376 23777 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:09.966921 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.963219 23932 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.973721 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:09.973850 23785 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:09.937804 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.959758 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.975056 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:09.960031 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:09.957806 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.952623 23781 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:09.974623 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:09.976701 23780 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.977636 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:09.964462 26490 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:09.979206 26698 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:09.980108 26700 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:09.958688 26697 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:09.959049 26675 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.983269 26675 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:09.985925 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 123319us
I20260709 12:14:09.984586 26708 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:09.986619 23825 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.086s (SENT)
I20260709 12:14:09.975551 23776 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:09.959172 26491 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:09.989295 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.997900 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:09.942972 26676 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:14:09.999820 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:09.980130 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.000970 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.941110 26520 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.000522 23776 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:09.946107 25172 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:09.952649 26673 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.010215 26714 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:10.015115 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:10.017040 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59314us
I20260709 12:14:09.972405 26602 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:09.978197 23681 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:09.952523 26711 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.940835 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:09.953250 26306 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.955448 26587 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.024652 25177 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:09.980371 26490 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:09.990536 26641 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.028122 26712 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.946962 26572 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:09.986750 26698 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:09.990538 26649 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:10.028888 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:09.998704 26651 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:10.034523 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:10.035990 23780 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:10.004887 26709 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:10.056638 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:10.058866 26373 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:10.059386 26708 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:10.059989 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.060235 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.060557 23859 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.060686 26708 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:10.061080 26708 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:10.061196 23851 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:10.061401 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.062003 26708 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.018498 26692 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.063135 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.063722 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.065699 23852 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:10.061919 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.067975 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.068113 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.069221 23854 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:10.046705 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60104.8us
I20260709 12:14:09.985014 26657 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.046989 23681 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:10.048645 23785 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:14:10.071027 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.071623 23681 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:09.999444 26662 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:10.047456 26712 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:10.072721 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.116s (SENT)
I20260709 12:14:10.073601 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:10.053012 26706 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:14:10.074828 26657 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:10.053830 26689 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:09.977912 26715 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:14:10.035534 26669 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.075546 26373 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.075949 23681 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 6, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 6 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.079276 23898 leader_election.cc:400] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5.
I20260709 12:14:10.079741 23898 leader_election.cc:403] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:14:10.080358 26490 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5.
I20260709 12:14:10.067306 26708 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:10.036041 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 202199us
I20260709 12:14:10.030341 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:10.038965 26702 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:14:10.053189 26717 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.084405 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69918.2us
I20260709 12:14:10.084631 26717 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.085858 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.085096 26692 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.086512 23929 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.086788 23859 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:10.087193 26490 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:10.086719 26491 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:10.088696 26491 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.089636 26491 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.092362 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.095397 26713 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:10.096714 23825 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.205s (SENT)
I20260709 12:14:10.094564 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.102366 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.107138 26651 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:10.108764 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:10.108621 26651 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.109620 25172 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:10.054495 26720 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.127127 23928 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:10.114889 23825 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.209s (SENT)
I20260709 12:14:10.051564 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:10.120913 26383 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.044668 26711 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.056237 26709 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:10.045471 26714 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.048475 26587 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.128400 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:10.135506 23929 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:10.141583 23932 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:10.149209 23931 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:10.159754 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.139243 26694 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:10.162846 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.166369 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:10.168332 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 166186us
I20260709 12:14:10.123843 26692 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.169214 23929 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:10.173045 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 58204.6us
W20260709 12:14:10.178426 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 84841.3us
I20260709 12:14:10.178440 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.179092 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.179488 23859 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:10.179664 23854 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:10.180482 23856 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:10.182017 23854 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:10.054858 26484 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.152635 26709 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:10.189774 23859 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:10.155259 26407 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.150472 26649 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:10.162539 26694 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:10.201659 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.211410 23925 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:10.202090 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.202459 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.142756 26663 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.213198 23859 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
W20260709 12:14:10.229645 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 135163us
I20260709 12:14:10.176398 26383 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.242676 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.197917 25169 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:10.211246 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.243315 23932 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:10.243425 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.053014 26698 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.248153 23856 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:10.249102 26704 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:14:10.251698 26586 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.252880 26484 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:10.203413 26736 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:10.254076 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.255887 23929 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:10.258719 23925 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
W20260709 12:14:10.260125 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 48969.4us
I20260709 12:14:10.199734 26713 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.260820 23934 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:10.287675 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.288100 23934 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:10.289352 23856 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:10.290387 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47047.3us
I20260709 12:14:10.267371 23855 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:14:10.267995 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 239179us
I20260709 12:14:10.269615 26681 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.270531 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.292534 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.292826 23860 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:10.241595 26694 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.259099 26657 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:14:10.265125 26720 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:10.272116 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.268968 26663 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.269696 23859 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.271705 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.273559 23852 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:10.298125 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.298556 23851 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.275007 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:10.283161 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.284740 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.285399 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:10.299149 23853 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.299458 23929 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.285807 26727 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:10.302242 23926 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:10.286010 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.291698 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.294322 23858 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:10.298856 23927 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:10.175019 26602 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.302983 23925 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:10.282923 26407 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.306311 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.319835 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.293033 26667 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
W20260709 12:14:10.334455 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.302518 26698 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:10.348641 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080003a8e0 after lost signal to thread 22632
W20260709 12:14:10.352370 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080003a960 after lost signal to thread 23717
I20260709 12:14:10.287976 26586 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.305169 26720 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:10.367996 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61651.7us
I20260709 12:14:10.377462 23932 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:10.398588 23925 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:10.368018 26407 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.400699 23929 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:10.399255 23851 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:10.402359 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.376394 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080003b0e0 after lost signal to thread 23809
I20260709 12:14:10.372771 26651 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.409251 26720 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.413509 26717 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:10.307408 26727 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.406651 26407 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.408241 23928 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:10.416798 26586 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:10.418329 25168 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:10.411787 26713 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.405608 26587 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:10.412194 26723 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 12:14:10.445070 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44442.3us
I20260709 12:14:10.445809 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:10.427699 23714 debug-util.cc:398] Leaking SignalData structure 0x7b08003578c0 after lost signal to thread 23884
W20260709 12:14:10.448122 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080039a6a0 after lost signal to thread 23958
I20260709 12:14:10.426896 26415 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.452308 23714 debug-util.cc:398] Leaking SignalData structure 0x7b0800303d00 after lost signal to thread 23959
W20260709 12:14:10.436606 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 130393us
I20260709 12:14:10.452752 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:10.453590 23714 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8940 after lost signal to thread 25207
W20260709 12:14:10.430055 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.454128 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.406638 26383 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:14:10.302368 26714 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.474486 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.448684 23930 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:10.341038 26649 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.436213 26712 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:10.477736 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:10.477629 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.478610 23776 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:10.477625 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.481029 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:10.481848 23714 debug-util.cc:398] Leaking SignalData structure 0x7b08003b40a0 after lost signal to thread 26476
W20260709 12:14:10.475909 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73712.1us
I20260709 12:14:10.455220 26719 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 12:14:10.484136 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.456240 26590 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:10.485730 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.486821 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080036a7e0 after lost signal to thread 26584
I20260709 12:14:10.489485 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:10.474751 26731 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:10.495733 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:10.493793 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.476200 26651 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:14:10.474486 26415 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:10.513525 26734 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:14:10.515833 26733 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.500244 26681 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:10.515826 26730 raft_consensus.cc:2764] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:10.519603 26662 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:10.513222 26722 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.515712 26737 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.518553 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:10.514654 26407 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:10.514003 26735 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:14:10.560201 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.499097 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:10.580997 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 81989.2us
I20260709 12:14:10.581774 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.514633 26739 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:10.497181 26732 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:10.586987 26732 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:10.578660 26663 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:10.589552 26673 consensus_meta.cc:287] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.004s
I20260709 12:14:10.590010 26673 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.590420 23778 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:14:10.591348 26729 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.289s	user 0.004s	sys 0.000s
I20260709 12:14:10.594115 26663 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:10.594316 26725 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:14:10.548738 26717 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:14:10.484088 26716 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.483443 26572 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:10.548954 26738 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:14:10.631093 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:10.494546 26669 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:10.489460 26712 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:10.548642 26520 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:10.585958 26743 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:14:10.567710 26662 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.498211 26590 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.500672 26736 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:14:10.501844 26306 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.499437 26710 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.244s
I20260709 12:14:10.595876 26725 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:10.614426 26727 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:10.615978 26717 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:10.616995 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:10.617971 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:10.618791 26667 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:10.619477 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.620496 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 102037us
I20260709 12:14:10.621007 26719 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.621668 26724 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:10.587456 26732 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.635043 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.636984 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.635514 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:10.648526 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 152850us
I20260709 12:14:10.650192 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:10.696866 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59990.5us
I20260709 12:14:10.697081 23929 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:10.696399 23928 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:10.650404 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:10.602504 26681 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:10.662470 26712 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.661477 23898 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.674444 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:10.676278 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:10.676388 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.704842 23926 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:10.656347 26715 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:10.683053 26717 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:10.676520 26662 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.676688 26651 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:10.759830 26651 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:10.762194 26715 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:10.693037 25169 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:10.659555 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.698566 26667 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.763206 26715 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.764456 26667 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.691893 25172 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:10.702045 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.768394 26737 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.768740 23852 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.688879 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.716418 23853 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:10.682294 26725 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.770759 26725 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.771301 26698 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:10.761891 26717 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.772086 26730 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:14:10.715377 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.704295 25177 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:10.768101 26694 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:10.775079 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59809.8us
I20260709 12:14:10.695957 26669 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:10.767230 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 116878us
I20260709 12:14:10.696498 26673 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:10.773561 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 154151us
I20260709 12:14:10.781606 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.776945 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:10.721588 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.778035 26716 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:10.704066 26743 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:10.779547 25172 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.777776 23852 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:10.782996 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.778667 25177 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:10.787611 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.788756 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:10.789934 26747 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:14:10.791646 26749 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:10.794003 25113 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.968s (SENT) [suppressed 67 similar messages]
I20260709 12:14:10.708261 26383 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 12:14:10.711820 26733 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:10.795136 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.800812 23749 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:10.760162 26712 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:10.801126 23749 leader_election.cc:403] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:14:10.803447 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.807456 23851 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.813349 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.798753 26720 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.840217 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:10.844738 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 155932us
I20260709 12:14:10.795624 26730 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.800350 26649 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:10.865046 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:10.865736 23853 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.800203 26749 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:10.802318 26716 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.801556 26743 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:10.825021 26669 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.824136 26731 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.815583 26737 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.815178 26590 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.871958 26720 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.816552 26733 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.858295 26736 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 4. Current term is 5.
I20260709 12:14:10.860633 26724 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 12:14:10.885103 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 84413.1us
I20260709 12:14:10.912949 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:10.886219 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 185638us
I20260709 12:14:10.913936 26743 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.888913 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:10.894170 23853 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:10.864543 26649 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.824363 26407 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:10.864418 26491 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:10.874425 26709 consensus_meta.cc:287] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 12:14:10.868366 26722 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.863276 26673 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:10.863678 26712 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:10.905731 26720 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.906237 26749 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.921345 26736 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:10.906562 26733 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.977691 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:10.977716 23856 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:10.925206 26407 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:10.915297 26681 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.921346 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.923008 23855 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:10.924773 26649 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:10.928485 23859 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:10.901690 26708 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.008s
I20260709 12:14:10.947741 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:10.989955 23927 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
W20260709 12:14:10.950623 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61785.8us
I20260709 12:14:10.931685 23925 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:10.871904 26415 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:10.972402 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 178412us
I20260709 12:14:10.918432 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:10.869093 26667 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.920519 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:10.947654 26306 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:10.925522 26491 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:10.930450 26520 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:14:10.994357 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76003.5us
W20260709 12:14:10.995373 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 74952.8us
I20260709 12:14:10.976857 26731 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.996025 25113 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:11.002405 26744 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.002712 26748 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:10.947909 26712 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:11.013070 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:11.008832 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.009402 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.009636 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.015635 23781 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:11.015903 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.016192 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:11.010411 23784 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:11.010891 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:11.010912 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.019773 23783 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:10.913312 26716 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.931703 26709 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:10.953142 26673 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:10.930892 26722 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.019196 25177 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:11.026002 23854 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:11.005640 26724 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:11.031324 23781 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:11.006850 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.008458 26491 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:11.002429 26667 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.041714 23857 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:11.033823 26520 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.012925 26731 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.044194 26724 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:11.040413 26641 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:14:11.045858 26731 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.045624 26724 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:11.047065 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.023375 26306 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:11.024415 26415 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:11.026242 26592 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:11.073900 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 78350.5us
I20260709 12:14:11.077965 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:11.012593 26737 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:14:11.033746 26655 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:14:11.054118 26306 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.101400 26711 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.102583 26586 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:11.138126 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.138851 23780 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 3: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:11.109025 26720 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.071061 26694 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:11.109552 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.152391 23781 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:11.111871 26673 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.112567 26759 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:11.102777 26760 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:14:11.114190 26667 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.111821 26712 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.137225 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.166720 26736 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.167069 26712 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.168397 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:11.041143 26730 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:14:11.052078 26726 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:11.085772 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:11.173868 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94005.6us
I20260709 12:14:11.174646 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.176604 26694 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.177901 26694 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:11.080418 26415 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:11.135282 23783 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:11.158015 26655 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:11.155989 26722 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:11.158669 26709 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:11.051855 26407 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.137750 26586 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.159427 26725 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.075090 26640 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:14:11.160202 26663 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.138896 26737 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.136008 26306 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:11.164708 26762 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
I20260709 12:14:11.176990 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.192929 23852 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:14:11.159044 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.181937 23784 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.182960 23825 leader_election.cc:400] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
I20260709 12:14:11.194271 23825 leader_election.cc:403] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:11.193972 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.182574 26655 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:11.204541 25114 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:11.188485 26586 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.209496 26709 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
I20260709 12:14:11.212888 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.215286 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:11.176648 26662 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.216295 23930 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:11.188045 26748 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:11.216936 23856 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:11.195222 26592 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:11.204386 26722 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.207010 26727 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.237222 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.237468 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.238077 23780 raft_consensus.cc:2410] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:11.234176 23931 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:11.238710 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.184840 26736 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.239995 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:11.240504 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:11.243222 26736 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.245553 23780 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:14:11.235190 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54484.3us
I20260709 12:14:11.179683 26415 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.255681 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:11.257571 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 179783us
I20260709 12:14:11.195433 26663 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.195063 26737 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.232762 26592 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:11.244900 26634 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:11.270529 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62079.4us
I20260709 12:14:11.260604 26730 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.262676 26731 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:11.268355 26724 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:11.249881 26722 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:11.301980 26732 consensus_meta.cc:287] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.004s
I20260709 12:14:11.273550 26634 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.269692 26771 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.296473 26748 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.289072 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:11.280952 26736 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.280007 26720 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:14:11.261032 26727 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.342566 23784 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
W20260709 12:14:11.353379 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 97797.4us
I20260709 12:14:11.263043 26655 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.355044 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
W20260709 12:14:11.356092 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67108.8us
I20260709 12:14:11.356894 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.357317 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:11.358275 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:11.341686 26634 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.349323 26771 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.353296 26491 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.360689 25169 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:11.353654 26592 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.368923 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:11.367098 26771 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.380256 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.348227 26725 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:11.422612 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65799.4us
I20260709 12:14:11.382849 26697 raft_consensus.cc:2764] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:14:11.331506 26724 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.426924 23929 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:11.286432 26769 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:14:11.369081 26592 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.341979 26732 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.351336 26736 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.394166 26783 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:11.385120 26587 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:11.360682 26714 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:14:11.456142 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.457384 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:11.455132 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:11.435755 26655 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:11.465708 23932 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:11.458549 26775 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:14:11.446626 26771 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:14:11.486038 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:11.487427 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.489020 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.491550 23780 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 2.
I20260709 12:14:11.492192 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.460806 26709 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:11.470250 26725 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:11.467698 26726 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:11.494886 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:11.496428 26663 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.472412 26694 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:11.497308 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:11.500444 26782 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:14:11.502457 26694 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.500078 26726 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:11.504539 26726 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.464138 26759 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:14:11.509508 26641 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:11.510054 26641 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.510476 26641 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:11.510826 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.511664 26781 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:14:11.483194 26720 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.524190 26727 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.498263 26711 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:11.495229 26697 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:11.464390 26373 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:14:11.508714 26737 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.508051 26725 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:11.540091 26649 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.005s
I20260709 12:14:11.530259 26711 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:11.533213 26697 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:11.528916 26720 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.553529 26711 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:11.558809 26697 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:11.572129 26751 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.794s	user 0.000s	sys 0.005s
I20260709 12:14:11.573596 26732 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.574597 26736 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.545917 26649 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.567899 26724 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:11.582690 26720 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.587525 26651 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.589502 26651 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:11.592953 26641 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.592195 26306 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:11.593967 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.602118 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 314) took 1328 ms. Trace:
I20260709 12:14:11.602555 23857 rpcz_store.cc:276] 0709 12:14:10.274176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.274381 (+   205us) service_pool.cc:224] Handling call
0709 12:14:10.274651 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.042273 (+767622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.601861 (+559588us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54002df354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:11.602048 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763600256}
I20260709 12:14:11.606288 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.608263 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.608915 23857 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:11.610038 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:11.610606 23857 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.625113 26572 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.627526 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.636763 23859 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:11.637889 23859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:11.637933 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.638692 23855 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:11.645092 26689 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:11.645843 26689 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:11.652304 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:11.651881 26586 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.653569 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:11.654713 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:11.655158 26586 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.702611 23856 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:11.708132 23853 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:14:11.708902 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.709678 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:11.631748 26641 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.562076 26725 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:11.633945 26792 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:14:11.657347 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2534) took 1237 ms. Trace:
I20260709 12:14:11.748934 23851 rpcz_store.cc:276] 0709 12:14:10.420457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.432452 (+ 11995us) service_pool.cc:224] Handling call
0709 12:14:10.436450 (+  3998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.807952 (+371502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.657089 (+849137us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540037f854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:11.657279 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852739328}
I20260709 12:14:11.718585 26745 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:11.750640 26745 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.751056 26745 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:11.616129 26663 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:14:11.659083 26649 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.630483 26306 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:11.664739 26689 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.667506 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:11.667977 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:11.656145 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:11.664805 26750 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:11.668766 25172 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:11.669553 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.670279 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:11.658962 23858 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:11.763629 23784 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.671391 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:11.764586 23928 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:11.671576 23855 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:11.656759 26722 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:11.676344 26718 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.711246 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.713949 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:11.716188 26736 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.748409 23856 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:11.698249 26761 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.783254 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69377.8us
I20260709 12:14:11.784312 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:11.748309 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.760051 23857 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:11.759518 23751 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:11.763532 23779 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:14:11.768960 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 113129us
I20260709 12:14:11.786806 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:11.786986 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:11.777374 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:11.788681 23784 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:11.755288 26748 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.792892 25177 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:11.748610 26737 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.803504 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:11.812016 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:11.812740 23928 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.763216 26750 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:11.910451 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 123710us
I20260709 12:14:11.758030 26720 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:14:11.804517 26649 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.753738 26634 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.790333 26725 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.953702 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:11.806041 26689 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:11.801231 26726 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:11.808665 26727 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:11.808269 26722 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:11.940498 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 229288us
I20260709 12:14:11.815472 26669 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:11.960776 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.968226 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.962038 25176 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:14:11.968938 25176 pending_rounds.cc:85] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 5
I20260709 12:14:11.969216 25176 pending_rounds.cc:107] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:14:11.969450 25176 raft_consensus.cc:2889] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:11.923624 26783 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:11.943212 26724 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.976985 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.808487 26736 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.805824 26718 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:11.996352 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 63701.4us
W20260709 12:14:11.988303 26772 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.782s	user 0.006s	sys 0.000s
I20260709 12:14:11.964540 26726 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:11.980926 26803 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:11.909607 26407 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:14:11.816231 26748 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.816843 26761 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:11.988435 23776 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:11.963671 26732 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.967274 26722 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.906440 26737 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:11.989289 26711 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.965451 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:11.968122 26780 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:14:11.986620 23928 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:11.987114 26724 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:11.990286 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:11.955070 26716 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:14:12.032052 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:12.032616 23928 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:12.028080 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 62715.5us
I20260709 12:14:12.033411 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:12.008152 26692 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.029525 26726 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.025581 26722 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:11.988903 26750 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.039894 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.044129 23855 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:12.041440 23857 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:12.041709 23853 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:12.048800 23928 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:12.050107 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.012151 26663 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:12.052587 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.053110 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 262) took 1080 ms. Trace:
I20260709 12:14:12.053269 23851 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 3.
I20260709 12:14:12.053632 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.055739 23933 rpcz_store.cc:276] 0709 12:14:10.972575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.972738 (+   163us) service_pool.cc:224] Handling call
0709 12:14:10.973143 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.973487 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.973511 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.977021 (+  3510us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.205999 (+228978us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:11.207398 (+  1399us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.207636 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.207712 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.701831 (+494119us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:12.042214 (+340383us) raft_consensus.cc:1685] finished
0709 12:14:12.052905 (+ 10691us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.053048 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":450,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":6838,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":8153,"spinlock_wait_cycles":13576704,"thread_start_us":219513,"threads_started":1,"wal-append.queue_time_us":220404}
I20260709 12:14:12.035847 26709 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:12.059886 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:12.063541 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.030493 26520 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:14:12.070709 23856 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:12.071322 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.072189 23857 raft_consensus.cc:2410] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:12.074872 23855 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:12.028653 26732 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.112430 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:12.113106 23933 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:12.068147 26800 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:14:12.102001 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.061942 26709 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:12.117405 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.120961 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:12.039209 26803 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 12:14:12.129294 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.052103 26692 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.139752 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.079324 26586 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.080672 26789 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.120003 26709 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.146972 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:12.148542 26797 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:14:12.149641 26794 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:14:12.154774 23931 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:12.155683 26804 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:12.156173 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:12.156579 26796 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:14:12.158665 26520 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:12.159117 26736 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.159233 26520 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.161682 26520 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.156350 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2287) took 1878 ms. Trace:
I20260709 12:14:12.162665 23782 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:12.164150 23931 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:12.165182 23855 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:12.162227 23932 rpcz_store.cc:276] 0709 12:14:10.277931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.425253 (+147322us) service_pool.cc:224] Handling call
0709 12:14:10.427513 (+  2260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.469669 (+1042156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.156104 (+686435us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000e4f54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:12.156292 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387052416}
I20260709 12:14:12.167811 26692 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.166184 23927 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:12.175908 23782 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:12.131505 26634 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.180331 25114 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:12.185560 23928 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:14:12.190716 23779 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:12.192112 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:12.195807 26663 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:12.196403 26663 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:12.197285 26663 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.162346 26724 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.156258 26804 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:12.207199 26748 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:12.207782 26737 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.190039 26780 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:12.206208 26799 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:14:12.145308 26586 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.224232 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.228595 23852 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:12.212489 26804 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:12.215761 26697 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.001s
I20260709 12:14:12.236450 26662 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.250037 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.251049 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.251679 23858 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:12.252339 23928 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:12.252907 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2347) took 1015 ms. Trace:
W20260709 12:14:12.251674 26802 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:14:12.253822 23854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 305) took 2054 ms (client timeout 2155 ms). Trace:
W20260709 12:14:12.254179 23854 rpcz_store.cc:269] 0709 12:14:10.199653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.199799 (+   146us) service_pool.cc:224] Handling call
0709 12:14:12.253619 (+2053820us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:12.253717 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799027584}
I20260709 12:14:12.254743 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 348) took 1647 ms. Trace:
I20260709 12:14:12.255352 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.255935 23854 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:12.256699 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.257301 23852 raft_consensus.cc:2376] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7. Re-sending same reply.
I20260709 12:14:12.253288 23781 rpcz_store.cc:276] 0709 12:14:11.237414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.240258 (+  2844us) service_pool.cc:224] Handling call
0709 12:14:12.252744 (+1012486us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540035a2d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:12.252850 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261864192}
I20260709 12:14:12.255079 23778 rpcz_store.cc:276] 0709 12:14:10.606949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.607128 (+   179us) service_pool.cc:224] Handling call
0709 12:14:10.609756 (+  2628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.657293 (+ 47537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.657332 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:11.706631 (+1049299us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b2c00031b98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:11.716163 (+  9532us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.955622 (+239459us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:11.955657 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.955741 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.254446 (+298705us) raft_consensus.cc:1685] finished
0709 12:14:12.254466 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.254661 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70091,"raft.queue_time_us":71356,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":11923,"spinlock_wait_cycles":9651968,"thread_start_us":182032,"threads_started":1,"wal-append.queue_time_us":255744}
I20260709 12:14:12.260887 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.261489 23851 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:12.258225 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:12.262547 23852 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:12.262985 23932 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:12.272867 23854 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:12.277469 23858 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:12.227401 26780 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:12.279959 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 372475us
I20260709 12:14:12.271209 23930 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 3.
I20260709 12:14:12.177597 26789 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.282660 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:12.284962 23932 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:12.266166 26655 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.233147 26718 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.309710 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:12.311553 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 119534us
I20260709 12:14:12.274240 26761 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:12.267274 26717 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.305500 23852 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:12.306268 23851 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 4.
I20260709 12:14:12.315340 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:12.296479 26803 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.322713 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:12.326310 23778 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:12.297071 26711 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.261097 26697 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.333685 23781 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:12.336050 23858 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:14:12.336498 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72410.5us
I20260709 12:14:12.336406 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.339293 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.340014 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:12.339864 23851 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:12.340969 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:12.320951 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.306854 26724 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:14:12.342538 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:12.309960 26655 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.316354 26713 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:14:12.366119 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:12.514840 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 172829us
W20260709 12:14:12.449168 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.538789 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.469545 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 273) took 1111 ms. Trace:
I20260709 12:14:12.416700 23933 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
W20260709 12:14:12.639133 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.644814 23851 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:12.647228 23778 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:12.653165 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.396982 26572 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:12.652863 23929 rpcz_store.cc:276] 0709 12:14:11.358187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.065648 (+707461us) service_pool.cc:224] Handling call
0709 12:14:12.065985 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:12.066315 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.066343 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:12.066776 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:14:12.129181 (+ 62405us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:12.327794 (+198613us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.469137 (+141343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.469239 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.469291 (+    52us) raft_consensus.cc:1685] finished
0709 12:14:12.469303 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.469468 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9755,"prepare.queue_time_us":356008,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":180183,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":176,"thread_start_us":374549,"threads_started":3,"wal-append.queue_time_us":133520}
I20260709 12:14:12.760000 23851 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:14:12.342465 26783 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.344343 26662 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:12.779233 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 130533us
I20260709 12:14:12.780287 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:12.556411 26714 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:12.452764 26812 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:14:12.337852 26803 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.799021 23855 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:12.761242 26737 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.487967 26732 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:12.761623 26748 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:12.773615 26694 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.760046 26572 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:12.797973 26722 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.738s	user 0.004s	sys 0.006s
I20260709 12:14:12.796641 26714 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.801708 26711 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:12.813078 26662 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:12.827584 26726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.019s	sys 0.040s
W20260709 12:14:12.854526 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 512016us
I20260709 12:14:12.366407 26780 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.836126 26602 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:12.840435 26714 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:12.794243 26783 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.864880 23855 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:12.865772 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:12.853695 26725 consensus_meta.cc:287] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.896s	user 0.011s	sys 0.000s
I20260709 12:14:12.815858 26737 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.874603 26602 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:12.867039 26713 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.874881 26657 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:12.875326 26713 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.874899 26823 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:12.875656 26602 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.876238 26823 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.876606 26823 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:12.877660 26713 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.878278 26809 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:12.878713 26809 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:12.853601 26586 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.862152 26663 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.828759 26694 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.861694 26789 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:12.875394 26657 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.840619 26722 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:12.829208 26748 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:12.887818 26756 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 1.877s	user 0.002s	sys 0.016s
W20260709 12:14:12.897019 26824 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:14:12.897708 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:12.868698 26709 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.683s	user 0.007s	sys 0.004s
I20260709 12:14:12.855695 26711 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:12.891453 26669 consensus_meta.cc:287] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.917s	user 0.005s	sys 0.005s
I20260709 12:14:12.904925 26828 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:12.905632 26662 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.906143 26657 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.906436 26783 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.879559 26809 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:12.886129 26745 consensus_meta.cc:287] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.132s	user 0.007s	sys 0.005s
W20260709 12:14:12.863715 26726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.019s	sys 0.040s
I20260709 12:14:12.880630 26697 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:12.928222 26649 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.973s	user 0.010s	sys 0.001s
W20260709 12:14:12.982524 26689 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.020s	user 0.010s	sys 0.001s
I20260709 12:14:12.982942 26649 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:12.908964 26713 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:14:12.984272 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:12.932808 26655 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:12.907302 26709 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.961645 26748 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:12.973491 26572 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.899453 26663 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:12.886361 26732 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:14:12.934140 26669 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:12.980072 26809 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:12.975059 26745 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:12.980695 26717 consensus_meta.cc:287] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.004s
I20260709 12:14:12.908785 26722 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:12.981483 25168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 216) took 2526 ms. Trace:
I20260709 12:14:12.929847 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2552) took 1279 ms. Trace:
I20260709 12:14:13.024024 23857 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:14:13.024638 23859 raft_consensus.cc:606] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:13.025153 23859 raft_consensus.cc:614] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:12.985406 23858 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:14:12.987396 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52243us
W20260709 12:14:13.012777 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.023454 23853 rpcz_store.cc:276] 0709 12:14:11.650608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.715749 (+ 65141us) service_pool.cc:224] Handling call
0709 12:14:11.716055 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.046106 (+330051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.929577 (+883471us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002d4bd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:12.929782 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748503296}
I20260709 12:14:13.008059 26748 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.027875 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:13.028569 23856 raft_consensus.cc:2368] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4.
I20260709 12:14:13.022866 26520 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.035912 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:13.039366 23853 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:13.018276 26828 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.026312 26759 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.027709 26826 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.034729 26657 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:13.033169 26789 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:13.036283 26725 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.042987 26711 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.038635 26713 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:14:13.038300 26748 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.043120 26520 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.050714 26828 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.103524 23853 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:14:13.106810 26827 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:13.057524 26826 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.023277 25168 rpcz_store.cc:276] 0709 12:14:10.480676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.482599 (+  1923us) service_pool.cc:224] Handling call
0709 12:14:10.574274 (+ 91675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.648214 (+ 73940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.656587 (+  8373us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.695760 (+ 39173us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.824740 (+128980us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:10.857502 (+ 32762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.863873 (+  6371us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.849861 (+1985988us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:12.980963 (+131102us) raft_consensus.cc:1685] finished
0709 12:14:12.981026 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.981381 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24808,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13314,"spinlock_wait_cycles":27868928,"thread_start_us":106145,"threads_started":1,"wal-append.queue_time_us":235632}
I20260709 12:14:13.110790 26826 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.063962 26657 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.050598 26759 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.113358 26657 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:13.113970 26657 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.114410 26657 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.068881 26725 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:13.121548 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.067353 26832 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:13.083809 26831 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:13.092120 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55317.7us
I20260709 12:14:13.126103 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:13.122400 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:13.127884 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.068385 26789 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.129020 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:13.100260 26737 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:13.121673 26694 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:13.082751 26649 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:14:13.127149 26832 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.143756 26663 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:13.132552 26837 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:14:13.132747 26826 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:13.130173 26834 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
W20260709 12:14:13.147202 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 123771us
I20260709 12:14:13.155242 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.155547 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.307181 23779 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:13.156925 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.157266 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.312649 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:13.158417 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.314109 23783 raft_consensus.cc:2410] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:14:13.159490 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.315073 23780 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:13.178678 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.145130 26841 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:13.128660 26831 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:13.199904 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.146095 26709 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.356597 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.140262 26697 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:13.307183 23781 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:13.442587 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:13.254830 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.360114 23778 raft_consensus.cc:2376] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6. Re-sending same reply.
I20260709 12:14:13.153853 26832 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:13.360349 26826 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:13.360921 26694 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.361578 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.144994 26823 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.438541 26689 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.201076 26717 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.292s
I20260709 12:14:13.360617 23780 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:14:13.154042 26649 consensus_queue.cc:1243] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:13.447976 23781 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 3.
I20260709 12:14:13.478989 23785 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:13.359457 26733 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.181s	user 0.055s	sys 0.062s
I20260709 12:14:13.488129 23779 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:13.150307 26737 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.200719 26373 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.271s
I20260709 12:14:13.549522 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.556576 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.562999 23785 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:13.568814 23778 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:13.479717 26851 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.471009 26731 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.555754 26733 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.434s	user 0.055s	sys 0.062s
I20260709 12:14:13.472815 26709 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.446688 26697 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:13.597100 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 292078us
I20260709 12:14:13.565073 26850 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:13.571821 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.602290 23782 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:13.442924 26843 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.574815 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 331) took 1325 ms. Trace:
W20260709 12:14:13.603780 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 457204us
I20260709 12:14:13.540416 26694 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:13.600095 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:13.555655 26689 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.571165 23779 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:13.580468 26717 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:13.612356 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:13.570631 23785 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:13.604090 23852 rpcz_store.cc:276] 0709 12:14:12.249063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.362861 (+113798us) service_pool.cc:224] Handling call
0709 12:14:12.363349 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:12.540391 (+177042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.540463 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:12.761107 (+220644us) log.cc:844] Serialized 26 byte log entry
0709 12:14:13.011950 (+250843us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:13.011993 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.012068 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:13.574583 (+562515us) raft_consensus.cc:1685] finished
0709 12:14:13.574602 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.574757 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174408,"raft.queue_time_us":175586,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":157,"thread_start_us":199526,"threads_started":1,"wal-append.queue_time_us":249873}
I20260709 12:14:13.614837 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.576524 26491 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:13.618870 25175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 689) took 1351 ms (client timeout 1708 ms). Trace:
W20260709 12:14:13.629390 25175 rpcz_store.cc:269] 0709 12:14:12.267457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.597038 (+1329581us) service_pool.cc:224] Handling call
0709 12:14:13.618789 (+ 21751us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:13.574101 26737 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.636346 23782 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:13.643376 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:13.639747 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.640689 26407 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.587304 26731 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.595266 26709 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.612727 26717 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:13.622843 26745 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.637197 26587 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:13.706779 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:13.707054 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.030s	sys 0.022s
I20260709 12:14:13.723714 26828 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:13.641382 26748 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:13.727094 26847 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:14:13.585470 26851 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:13.973954 26861 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:13.998020 26851 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:13.591523 26831 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.998554 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2565) took 1091 ms. Trace:
W20260709 12:14:13.998627 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 130349us
I20260709 12:14:13.998957 26831 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:13.999579 26711 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.000090 26711 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.607142 26826 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.002190 26711 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:13.703768 26572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.023s	sys 0.039s
I20260709 12:14:13.859627 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.706065 26649 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.612694 26848 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:14:13.715432 26587 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.627260 26373 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:14:14.003553 26587 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:14.002828 26572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.029s	sys 0.045s
I20260709 12:14:13.617849 26837 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:14:14.005316 26649 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.617614 26843 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.875105 23779 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:14.007573 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:13.872854 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:13.893287 26689 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:13.896260 26853 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:14:14.009074 25175 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:13.896919 26856 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:14.010337 26856 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.011859 23782 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:14.012624 23785 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 12:14:14.012970 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2567) took 1076 ms. Trace:
I20260709 12:14:13.897446 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47418 (request call id 20) took 2385 ms. Trace:
I20260709 12:14:13.899362 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 324) took 1652 ms. Trace:
I20260709 12:14:14.008708 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:13.982695 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:13.886997 26809 consensus_meta.cc:287] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.861s	user 0.008s	sys 0.008s
W20260709 12:14:13.907706 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.971534 23926 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:14.016839 23856 rpcz_store.cc:276] 0709 12:14:12.247196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.249784 (+  2588us) service_pool.cc:224] Handling call
0709 12:14:13.899120 (+1649336us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54004367d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:13.899296 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788501248}
I20260709 12:14:14.011467 26843 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.008561 26649 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.980892 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:13.993912 26850 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:13.990113 26870 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:14.018868 23932 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:13.976183 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:14.019917 26870 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.995497 26709 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:13.997077 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.039s	sys 0.022s
I20260709 12:14:14.020424 26850 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.022760 26870 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.997821 26861 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:13.663576 26520 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:13.871416 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:13.981175 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:13.875036 26839 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.423s	user 0.000s	sys 0.007s
I20260709 12:14:13.935883 26717 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:13.703616 26407 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:13.970036 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.018452 23856 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:14.013855 23859 rpcz_store.cc:276] 0709 12:14:11.512494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.637649 (+125155us) service_pool.cc:224] Handling call
0709 12:14:13.897184 (+2259535us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400413cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7ac853 00007fb2b277a60a 00007fb2abd007d4 00007fb2abd00762 00007fb2abd00691 00007fb2abd0060c 00007fb2abcff8c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:13.897330 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3187595136}
W20260709 12:14:14.022770 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.013327 23858 rpcz_store.cc:276] 0709 12:14:12.936345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.108382 (+172037us) service_pool.cc:224] Handling call
0709 12:14:13.108803 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:13.127870 (+ 19067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.127904 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:13.129875 (+  1971us) log.cc:844] Serialized 52 byte log entry
0709 12:14:13.362067 (+232192us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:13.362106 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.362171 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.012753 (+650582us) raft_consensus.cc:1685] finished
0709 12:14:14.012774 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.012910 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22042,"raft.queue_time_us":20694,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8728,"spinlock_wait_cycles":124160,"thread_start_us":218378,"threads_started":1,"wal-append.queue_time_us":463278}
I20260709 12:14:13.998903 23853 rpcz_store.cc:276] 0709 12:14:12.907403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.105296 (+197893us) service_pool.cc:224] Handling call
0709 12:14:13.105656 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.105876 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:13.541445 (+435569us) log.cc:844] Serialized 14 byte log entry
0709 12:14:13.974209 (+432764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:13.974272 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:13.974317 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.974383 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.998493 (+ 24110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12893,"prepare.queue_time_us":516379,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":43404,"raft.run_cpu_time_us":15425,"raft.run_wall_time_us":60139,"thread_start_us":720726,"threads_started":3}
I20260709 12:14:13.976271 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:14.026006 23929 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
W20260709 12:14:13.999991 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.014418 23779 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:13.885264 26789 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:14.012475 26856 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.036527 25175 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:14.038673 23932 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:14.077098 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:14.078644 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:14.086629 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:14.087322 23779 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:14.089637 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:14.082423 26837 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:14.081871 26870 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:14.093115 23928 raft_consensus.cc:2410] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:14:14.084415 23925 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:14.046496 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:14.096576 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50149.9us
I20260709 12:14:14.025350 26731 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:14.101964 23925 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:14.101760 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:14.047950 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:13.991688 26491 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.047412 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:14.044663 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:14.110061 23929 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:14.111279 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:14.114022 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:14.032310 25168 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:14.080454 26709 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.095799 26737 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.115502 23778 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:14.028641 26858 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.082144 26689 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:14:14.103252 26717 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.102057 26745 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:14.116232 26714 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.242s	user 0.006s	sys 0.007s
I20260709 12:14:14.126255 23778 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:14.080572 26520 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:14:14.092206 26837 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.115422 26828 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.134763 23856 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:14.098852 26870 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.128578 26745 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.123240 26491 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.159296 23926 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:14.160466 26491 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:14.124500 26709 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.151753 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 459) took 1886 ms. Trace:
I20260709 12:14:14.154151 26861 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.155783 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:14.138389 26789 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.121687 26587 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.168318 26709 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:14.145207 26849 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:14:14.170255 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:14.163544 25176 rpcz_store.cc:276] 0709 12:14:12.265728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.265851 (+   123us) service_pool.cc:224] Handling call
0709 12:14:12.266197 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:12.266556 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.266584 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:13.894669 (+1628085us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b180048b9e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a
0709 12:14:13.895059 (+   390us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b2c000e5e48. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:13.902611 (+  7552us) log.cc:844] Serialized 52 byte log entry
0709 12:14:14.126285 (+223674us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:14.126327 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.126397 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.151482 (+ 25085us) raft_consensus.cc:1685] finished
0709 12:14:14.151503 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.151688 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6761,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":22293120,"thread_start_us":151180,"threads_started":1,"wal-append.queue_time_us":201811}
I20260709 12:14:14.158229 23781 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:14.149530 26745 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.180132 23856 raft_consensus.cc:2410] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:14:14.175369 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 63993.5us
W20260709 12:14:14.181305 26871 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:14:14.181856 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:14.182603 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2313) took 1023 ms. Trace:
I20260709 12:14:14.182963 23783 rpcz_store.cc:276] 0709 12:14:13.159398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.360706 (+201308us) service_pool.cc:224] Handling call
0709 12:14:13.360990 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.611463 (+250473us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:13.630658 (+ 19195us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.638852 (+  8194us) log.cc:844] Serialized 26 byte log entry
0709 12:14:13.994280 (+355428us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:13.994322 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.994389 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.182405 (+188016us) raft_consensus.cc:1685] finished
0709 12:14:14.182423 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.182545 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95079,"raft.queue_time_us":548379,"raft.run_cpu_time_us":3585,"raft.run_wall_time_us":40160,"thread_start_us":282369,"threads_started":2,"wal-append.queue_time_us":402804}
I20260709 12:14:14.184705 26861 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:14.185034 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:14.132944 26714 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.144527 26667 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:14:14.128288 26858 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.190119 26491 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:14.196954 23781 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:14.192400 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:14.202147 25168 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:14.190472 26873 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.189659 26861 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:14.205091 26860 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:14:14.225400 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2568) took 1194 ms. Trace:
I20260709 12:14:14.225857 23855 rpcz_store.cc:276] 0709 12:14:13.031011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.146443 (+115432us) service_pool.cc:224] Handling call
0709 12:14:13.146753 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.153188 (+  6435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.153220 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.153692 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.203861 (+1050169us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b180007efa0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:14:14.217044 (+ 13183us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:14.217091 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.217161 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.218408 (+  1247us) raft_consensus.cc:1685] finished
0709 12:14:14.218431 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.225325 (+  6894us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6083,"raft.queue_time_us":155395,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":54806,"spinlock_wait_cycles":7825280,"thread_start_us":12976,"threads_started":1,"wal-append.queue_time_us":913108}
I20260709 12:14:14.177794 26789 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:14.219405 26874 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:14:14.237432 26837 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:14.239735 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.205037 26491 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.256210 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:14.257913 26491 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.262833 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2569) took 1156 ms. Trace:
I20260709 12:14:14.263512 26789 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.265117 26407 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.266624 26587 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:14.214128 26873 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.263298 23852 rpcz_store.cc:276] 0709 12:14:13.106737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.626302 (+519565us) service_pool.cc:224] Handling call
0709 12:14:13.626669 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.652413 (+ 25744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.652447 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.703899 (+ 51452us) log.cc:844] Serialized 26 byte log entry
0709 12:14:13.977642 (+273743us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:13.977682 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.977748 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.262609 (+284861us) raft_consensus.cc:1685] finished
0709 12:14:14.262628 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.262765 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160795,"raft.queue_time_us":25973,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":42828,"thread_start_us":103369,"threads_started":1,"wal-append.queue_time_us":241455}
I20260709 12:14:14.270380 26873 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.278347 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:14.278940 23852 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:14.288050 23781 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:14.300985 23781 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:14:14.279856 26698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.124s	user 0.014s	sys 0.018s
W20260709 12:14:14.305276 26698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.150s	user 0.014s	sys 0.018s
I20260709 12:14:14.336532 26750 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:14.341501 23783 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:14:14.343297 26832 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.825s	user 0.003s	sys 0.010s
I20260709 12:14:14.350242 23852 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:14.351673 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:14.378908 26880 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.188s	user 0.000s	sys 0.004s
I20260709 12:14:14.361465 26832 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.400213 23784 raft_consensus.cc:2368] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:14.401350 26634 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:14.402405 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2308) took 1245 ms. Trace:
I20260709 12:14:14.402107 26634 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.402778 23784 rpcz_store.cc:276] 0709 12:14:13.156610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.156707 (+    97us) service_pool.cc:224] Handling call
0709 12:14:14.402186 (+1245479us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400092c54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:14.402341 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501348224}
I20260709 12:14:14.404006 23778 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:14.390965 26725 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:14.425835 26789 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.430711 26789 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.434407 26572 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:14.435212 26572 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:14.440708 26745 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:14.442582 26788 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:14.443145 26788 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:14.446897 26814 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 1.906s	user 0.000s	sys 0.017s
I20260709 12:14:14.404590 26832 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:14.433812 26789 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:14.689544 23781 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:14:14.522966 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.723122 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2361) took 1018 ms. Trace:
I20260709 12:14:14.723621 23781 rpcz_store.cc:276] 0709 12:14:13.704433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.301808 (+597375us) service_pool.cc:224] Handling call
0709 12:14:14.689186 (+387378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.722791 (+ 33605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.723065 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:14.528585 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.488276 23777 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:14:14.521263 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.499238 23776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2339) took 3473 ms (client timeout 3167 ms). Trace:
W20260709 12:14:14.743875 23776 rpcz_store.cc:269] 0709 12:14:11.009046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.009173 (+   127us) service_pool.cc:224] Handling call
0709 12:14:14.482185 (+3473012us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54004263d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:14.482297 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237631360}
W20260709 12:14:15.181387 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.250480 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.546558 23749 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.005s	sys 0.052s
W20260709 12:14:15.589730 23749 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.005s	sys 0.055s
W20260709 12:14:15.590111 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 897103us
W20260709 12:14:15.031944 23860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.020s	sys 0.059s
W20260709 12:14:15.160369 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.245730 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.009s
W20260709 12:14:14.790740 23927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 267) took 3388 ms (client timeout 1882 ms). Trace:
I20260709 12:14:14.746526 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2324) took 1389 ms. Trace:
W20260709 12:14:14.793550 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.587893 25112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.008s	sys 0.021s
W20260709 12:14:15.597388 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.597962 23751 connection.cc:256] RPC call timeout handler was delayed by -2.0979327360000752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 12:14:14.987406 26804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.004s	sys 0.036s
I20260709 12:14:14.582773 26745 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:14.650336 26748 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.358299 26694 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:14:15.606796 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.002s	sys 0.015s
I20260709 12:14:15.611052 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 270) took 4252 ms. Trace:
W20260709 12:14:15.624745 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 880430us
W20260709 12:14:15.606980 23927 rpcz_store.cc:269] 0709 12:14:11.356342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.356459 (+   117us) service_pool.cc:224] Handling call
0709 12:14:14.744756 (+3388297us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54000f91d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:14.744862 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1831798656}
I20260709 12:14:15.607159 23778 rpcz_store.cc:276] 0709 12:14:13.357138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.131336 (+774198us) service_pool.cc:224] Handling call
0709 12:14:14.403843 (+272507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.404396 (+   553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.746314 (+341918us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400092c50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:14.746454 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626009856}
W20260709 12:14:15.608165 25112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.008s	sys 0.021s
W20260709 12:14:15.605736 23860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.037s	sys 0.094s
I20260709 12:14:14.918177 26572 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:14.826792 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 688) took 2661 ms. Trace:
I20260709 12:14:14.953522 26717 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:14:15.606889 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 1.25522e+06us
I20260709 12:14:14.857193 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:14.798604 26634 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:14.601608 26749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.452s	user 0.013s	sys 0.042s
I20260709 12:14:15.657732 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:14.791276 26858 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:15.609694 26804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.024s	sys 0.051s
I20260709 12:14:15.638737 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:14.752969 26714 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.643230 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.633821 26681 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.607867 26759 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:15.645201 26861 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:15.645337 26813 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.625149 23934 rpcz_store.cc:276] 0709 12:14:11.357220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.357345 (+   125us) service_pool.cc:224] Handling call
0709 12:14:11.439951 (+ 82606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:11.440336 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:11.440363 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:11.444568 (+  4205us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.691234 (+246666us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:11.691275 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.691348 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.157609 (+466261us) raft_consensus.cc:1685] finished
0709 12:14:12.157628 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.595681 (+3438053us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b540006bdd0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:15.609324 (+ 13643us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1245,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":6425,"spinlock_wait_cycles":127175936,"thread_start_us":208606,"threads_started":1,"wal-append.queue_time_us":298709,"wal-append.run_cpu_time_us":2826,"wal-append.run_wall_time_us":1383428}
I20260709 12:14:15.607074 26716 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:15.647909 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.081453 26789 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:15.608013 26730 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:15.621454 26724 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:15.633489 26694 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.666908 25175 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:15.644457 26761 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.634429 25172 rpcz_store.cc:276] 0709 12:14:12.165300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.165426 (+   126us) service_pool.cc:224] Handling call
0709 12:14:12.165794 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:12.166172 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.166223 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:12.166766 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:14:12.320027 (+153261us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:12.329537 (+  9510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.338259 (+  8722us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.446168 (+2107909us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:14.717940 (+271772us) raft_consensus.cc:1685] finished
0709 12:14:14.752837 (+ 34897us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.826685 (+ 73848us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1110,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"spinlock_wait_cycles":24404224,"thread_start_us":140797,"threads_started":1,"wal-append.queue_time_us":277424,"wal-append.run_cpu_time_us":20523,"wal-append.run_wall_time_us":3175204}
W20260709 12:14:15.670790 23959 heartbeater.cc:646] Failed to heartbeat to 127.22.25.126:39113 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.22.25.126:39113 timed out after 15.000s (SENT)
I20260709 12:14:15.633597 26745 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:15.673741 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:15.641149 26826 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:15.640908 26717 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:15.677726 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:15.706318 25168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 465) took 1434 ms. Trace:
W20260709 12:14:15.652374 26749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.503s	user 0.059s	sys 0.111s
I20260709 12:14:15.689154 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.642212 26831 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:15.693761 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:15.698643 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.705647 23784 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:15.628491 26748 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.672289 26858 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.708630 23779 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:15.652339 26572 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:14:15.714020 23781 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:15.718473 23780 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:15.710825 23778 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:15.719565 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:15.706957 26634 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.678944 26681 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.708184 26804 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.699618 26861 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.708467 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:15.708922 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.707114 26813 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.709347 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.716327 26789 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.711087 26826 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.712047 25168 rpcz_store.cc:276] 0709 12:14:14.271884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.279795 (+  7911us) service_pool.cc:224] Handling call
0709 12:14:15.697658 (+1417863us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.798074 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.710521 26745 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.812438 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.822113 23853 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:15.723204 26858 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.709543 26761 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.718647 26831 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.816327 26803 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.711272 26724 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.708770 26694 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.819068 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.861310 23852 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:15.862404 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.820451 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.863978 23856 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:15.865510 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.866400 23852 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:15.708464 26730 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.711709 26717 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.803233 26804 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.711687 26716 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:15.825623 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:15.836300 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.877866 23853 raft_consensus.cc:2410] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:14:15.878923 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:15.867177 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.799494 26634 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:15.824295 26639 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.036s	sys 0.042s
I20260709 12:14:15.871939 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:15.880278 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0305011340001329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
W20260709 12:14:15.880527 26639 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.036s	sys 0.043s
W20260709 12:14:15.880721 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0134011340001052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
I20260709 12:14:15.881130 23852 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:15.881989 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.882135 26813 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.882211 26749 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.882576 23852 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:15.882891 26749 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.883740 26759 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:15.884075 26804 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.880828 26789 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:15.884246 26759 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.884927 26789 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.885026 26749 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.885533 26749 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.885432 26789 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.886229 26759 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.886004 26749 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.886739 26759 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:15.887768 26716 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.889833 26803 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:15.890234 26803 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:15.890612 26803 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.891456 26789 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:15.892078 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.895632 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.883455 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:15.899950 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.900720 23853 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:15.901749 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.902603 23853 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:15.897689 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.903779 23852 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:15.869290 26771 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:15.908818 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:15.887203 26759 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.911479 26826 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:15.914690 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.916190 23853 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
W20260709 12:14:15.915570 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.917310 23852 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:15.911360 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:15.897945 26717 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:15.899322 26804 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:15.932858 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2315) took 1951 ms. Trace:
W20260709 12:14:15.932901 23751 connection.cc:256] RPC call timeout handler was delayed by -1.6280236810000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 12:14:15.968329 23751 connection.cc:256] RPC call timeout handler was delayed by -1.4903236810000635s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 12:14:15.967543 23852 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:14:15.971699 23751 connection.cc:256] RPC call timeout handler was delayed by -1.1267236810001577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
W20260709 12:14:15.941463 26896 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:14:15.927843 26730 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:15.941227 23931 rpcz_store.cc:276] 0709 12:14:13.981599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.981719 (+   120us) service_pool.cc:224] Handling call
0709 12:14:13.990287 (+  8568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.990710 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:14.029159 (+ 38449us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.918782 (+889623us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b180012b140. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:15.867437 (+948655us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:15.868040 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:14:15.868211 (+   171us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:15.868245 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.868314 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.925878 (+ 57564us) raft_consensus.cc:1685] finished
0709 12:14:15.932488 (+  6610us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.932743 (+   255us) inbound_call.cc:177] Queueing success response
0709 12:14:15.932377 (+  -366us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180012b140. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5557 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
Metrics: {"mutex_wait_us":6493,"prepare.queue_time_us":38264,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":41051,"raft.queue_time_us":212,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":37394,"spinlock_wait_cycles":5791488,"thread_start_us":277332,"threads_started":1,"wal-append.queue_time_us":1894819}
I20260709 12:14:15.926088 26718 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.928527 26841 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:15.982059 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:15.996173 23852 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:14:15.988369 23898 connection.cc:256] RPC call timeout handler was delayed by -1.1589268939999329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
I20260709 12:14:15.992530 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:15.994143 23859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.277s	user 0.020s	sys 0.016s
W20260709 12:14:15.934158 26895 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 12:14:15.998067 23859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.020s	sys 0.017s
I20260709 12:14:16.001080 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.001981 23929 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:16.006569 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.007395 23925 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:16.008157 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.010340 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.011195 23929 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:16.011868 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.019186 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:16.019711 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 137884us
I20260709 12:14:15.922830 26771 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:15.942401 26826 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:14:15.968998 26804 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:15.979568 26809 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:14:15.979529 26813 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.113s
I20260709 12:14:16.295980 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.299424 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.109841 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1003) took 5144 ms. Trace:
W20260709 12:14:16.104174 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 51413.5us
I20260709 12:14:16.049327 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.104951 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.275660 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:16.318416 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 326313us
W20260709 12:14:16.110865 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.149979 23927 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:16.298542 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.049026 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.049860 23930 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:16.019860 23932 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:16.020537 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:16.055447 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.317458 23925 raft_consensus.cc:2393] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:16.323120 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.323886 23931 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:16.358419 23933 raft_consensus.cc:2410] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:16.121021 26732 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.051908 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.390204 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.391911 23929 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:16.368943 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.276371 26910 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:16.318137 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:16.388669 23825 connection.cc:256] RPC call timeout handler was delayed by -2.92285744600008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 12:14:16.298980 26724 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:16.122817 26903 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:16.329120 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.048817 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:16.318069 26745 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.390115 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.390655 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.021390 26681 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:15.985759 26714 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.378s
I20260709 12:14:16.018864 26841 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.121413 26730 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.322s
I20260709 12:14:16.018610 26718 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.150681 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.388533 26761 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:16.324004 26858 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.517118 26919 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.360915 26694 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.518391 23931 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
W20260709 12:14:16.518749 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 200709us
W20260709 12:14:16.518033 23825 connection.cc:256] RPC call timeout handler was delayed by -2.831957445999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 12:14:16.583818 23825 connection.cc:256] RPC call timeout handler was delayed by -2.7806574460000775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 12:14:16.584169 23825 connection.cc:256] RPC call timeout handler was delayed by -2.535857445999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
W20260709 12:14:16.584471 23825 connection.cc:256] RPC call timeout handler was delayed by -2.5124574459998712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 12:14:16.584764 23825 connection.cc:256] RPC call timeout handler was delayed by -2.4854574460000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 12:14:16.585055 23825 connection.cc:256] RPC call timeout handler was delayed by -2.4521574460000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
W20260709 12:14:16.585376 23825 connection.cc:256] RPC call timeout handler was delayed by -2.4323574459999691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:14:16.585645 23825 connection.cc:256] RPC call timeout handler was delayed by -2.2230627129999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 12:14:16.585923 23825 connection.cc:256] RPC call timeout handler was delayed by -2.1758574460000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24760000000000001s for a response
W20260709 12:14:16.586191 23825 connection.cc:256] RPC call timeout handler was delayed by -1.8623235929999282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:14:16.586454 23825 connection.cc:256] RPC call timeout handler was delayed by -1.7948450169999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 12:14:16.386824 26749 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.056506 26904 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.498385 26921 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.520447 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.591498 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.435348 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:16.517863 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:16.521390 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.052152 26906 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:16.317693 23860 rpcz_store.cc:276] 0709 12:14:10.918874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.919055 (+   181us) service_pool.cc:224] Handling call
0709 12:14:10.925034 (+  5979us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:10.931277 (+  6243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:11.609288 (+678011us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800236440. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf 00007fb2ad7cc038
0709 12:14:11.754520 (+145232us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.025505 (+270985us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:14.403687 (+2378182us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b2c00000448. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:14.404340 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 12:14:15.628483 (+1224143us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:15.927037 (+298554us) log.cc:844] Serialized 14 byte log entry
0709 12:14:15.928387 (+  1350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.929443 (+  1056us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.062816 (+133373us) raft_consensus.cc:1685] finished
0709 12:14:16.062840 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.063043 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36952,"prepare.queue_time_us":1080534,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":97,"raft.queue_time_us":202361,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":42043,"spinlock_wait_cycles":26275584,"thread_start_us":1778129,"threads_started":5,"wal-append.queue_time_us":1502040,"wal-append.run_cpu_time_us":1398,"wal-append.run_wall_time_us":1030871}
I20260709 12:14:16.523191 26903 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.055543 26748 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.022404 26709 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.523129 26634 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.525748 26697 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.526999 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.517432 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:16.603871 23934 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:16.584825 26919 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.607002 26919 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.608920 26903 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.444204 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.482272 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.052856 26711 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.595033 26904 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.610074 26711 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.611522 26904 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:16.598255 26803 consensus_meta.cc:287] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
I20260709 12:14:16.612411 26803 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.612766 26711 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.213640 26907 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:16.613940 26907 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.615005 23853 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:16.619181 25169 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:16.432725 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.518877 26842 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:16.579907 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2322) took 3362 ms. Trace:
I20260709 12:14:16.580747 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1019) took 4376 ms. Trace:
I20260709 12:14:16.625355 23932 raft_consensus.cc:2410] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:16.519398 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.587567 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.588842 23898 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:16.580058 26858 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.395767 26732 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.597083 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.652554 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.652843 23852 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:16.653995 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.601413 26906 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.614490 26803 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.620023 26759 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.484030 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.622237 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.614950 23928 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
W20260709 12:14:16.623031 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.008s	sys 0.007s
I20260709 12:14:16.623926 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:16.485214 26911 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:16.625568 23857 rpcz_store.cc:276] 0709 12:14:12.204674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.204816 (+   142us) service_pool.cc:224] Handling call
0709 12:14:12.205152 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.024434 (+819282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.568972 (+3544538us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400413cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:16.580656 (+ 11684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882365696}
I20260709 12:14:16.626398 23785 rpcz_store.cc:276] 0709 12:14:13.207524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.616040 (+408516us) service_pool.cc:224] Handling call
0709 12:14:14.012380 (+396340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:14.013021 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.569809 (+2556788us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b5400093650. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:16.569945 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910653184}
W20260709 12:14:16.655189 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67678.7us
I20260709 12:14:16.656071 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.694167 23932 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:16.670837 23934 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:16.695494 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.696533 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.696870 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:16.697877 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.008s	sys 0.011s
W20260709 12:14:16.698280 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 719774us
I20260709 12:14:16.699025 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.702147 23930 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:16.699899 23934 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:16.593318 26912 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 12:14:16.703789 23934 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:16.710183 23929 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:16.704790 23931 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
W20260709 12:14:16.623558 26844 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 3.059s	user 0.000s	sys 0.025s
I20260709 12:14:16.624791 23925 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:16.715214 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.657423 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:16.710953 23934 pending_rounds.cc:85] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 3
I20260709 12:14:16.717691 23934 pending_rounds.cc:107] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:16.717980 23934 raft_consensus.cc:2889] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:16.699548 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.551853 26910 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:16.712471 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 123700us
I20260709 12:14:16.715902 23932 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
W20260709 12:14:16.716535 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59209us
I20260709 12:14:16.705921 26904 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:16.720314 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.723511 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:16.729161 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:16.656293 26771 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:16.704766 25169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 461) took 2947 ms (client timeout 1567 ms). Trace:
I20260709 12:14:16.725914 26803 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
W20260709 12:14:16.732417 26913 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:14:16.725010 26759 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:16.735133 26841 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.735421 26732 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.740074 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.748102 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.740084 23932 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:16.749428 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.750676 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.750967 23932 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:16.722564 26930 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:14:16.744161 26717 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.817s	user 0.007s	sys 0.004s
I20260709 12:14:16.745734 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2556) took 4430 ms. Trace:
I20260709 12:14:16.623930 26709 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.722477 26907 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.751564 26930 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.755322 26930 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:16.741860 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.760669 23932 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:16.752143 23851 rpcz_store.cc:276] 0709 12:14:12.315099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.773004 (+457905us) service_pool.cc:224] Handling call
0709 12:14:12.773401 (+   397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:13.027201 (+253800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.027236 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:13.029705 (+  2469us) log.cc:844] Serialized 52 byte log entry
0709 12:14:13.133265 (+103560us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:13.133301 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.133372 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.620734 (+3487362us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:16.735677 (+114943us) raft_consensus.cc:1685] finished
0709 12:14:16.735701 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.745644 (+  9943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24271,"raft.queue_time_us":208043,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":86,"spinlock_wait_cycles":11850496,"thread_start_us":310459,"threads_started":2,"wal-append.queue_time_us":397725}
I20260709 12:14:16.751099 26910 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.718291 26923 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:16.747007 23928 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:14:16.747790 26759 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:16.692890 26858 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.763461 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.770025 25174 raft_consensus.cc:2410] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:14:16.726270 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.719709 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:16.751539 25171 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:16.727191 26842 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:16.772938 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 53353.3us
I20260709 12:14:16.774398 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:16.621680 26921 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:16.754204 26709 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.769078 26759 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:16.736140 26925 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:16.789989 26718 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.736516 26733 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:16.785095 26657 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.768983 26714 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:16.767256 26923 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.769203 26813 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.789165 26906 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.783885 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:16.808220 26925 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:16.792507 26717 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:16.794239 26858 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.797550 26921 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:16.779841 26803 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:16.819658 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.817830 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:16.799117 26826 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:16.774324 26907 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:16.797588 23782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2355) took 3180 ms (client timeout 2155 ms). Trace:
I20260709 12:14:16.802946 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:16.824360 23782 rpcz_store.cc:269] 0709 12:14:13.616792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.665195 (+ 48403us) service_pool.cc:224] Handling call
0709 12:14:16.797410 (+3132215us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b5400094054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:16.797508 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18226560}
I20260709 12:14:16.803429 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:16.808841 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2581) took 2563 ms. Trace:
W20260709 12:14:16.824878 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 50576.1us
W20260709 12:14:16.736290 25169 rpcz_store.cc:269] 0709 12:14:13.705461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.714789 (+  9328us) service_pool.cc:224] Handling call
0709 12:14:16.048621 (+2333832us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:14:16.652349 (+603728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14812928}
I20260709 12:14:16.826162 23859 rpcz_store.cc:276] 0709 12:14:14.245577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.245836 (+   259us) service_pool.cc:224] Handling call
0709 12:14:14.278597 (+ 32761us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:14.278884 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.278910 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:14.688531 (+409621us) log.cc:844] Serialized 52 byte log entry
0709 12:14:15.998582 (+1310051us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:15.998627 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.998693 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.808600 (+809907us) raft_consensus.cc:1685] finished
0709 12:14:16.808626 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.808770 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":23753,"thread_start_us":1309575,"threads_started":1,"wal-append.queue_time_us":1582028}
I20260709 12:14:16.832759 23825 leader_election.cc:400] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:16.818364 26657 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:16.835934 26657 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.836391 26657 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:16.830478 26906 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:14:16.830317 26789 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:16.840008 26789 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.840426 26789 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:16.843044 26831 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:16.811168 26714 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.823473 23853 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:16.798406 26930 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.811324 26923 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:16.817188 26813 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.830829 26803 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:16.844861 26854 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 2.816s	user 0.009s	sys 0.013s
I20260709 12:14:16.845199 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2310) took 3688 ms. Trace:
I20260709 12:14:16.845412 26803 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:16.846984 26863 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 3.206s	user 0.002s	sys 0.017s
I20260709 12:14:16.833838 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:16.822692 26925 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:16.813278 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 328) took 4564 ms. Trace:
I20260709 12:14:16.849097 23777 rpcz_store.cc:276] 0709 12:14:13.157135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.157260 (+   125us) service_pool.cc:224] Handling call
0709 12:14:14.482477 (+1325217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.737367 (+254890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.844974 (+2107607us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54004263d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:16.845122 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3046855552}
I20260709 12:14:16.848927 23854 rpcz_store.cc:276] 0709 12:14:12.248353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.283912 (+ 35559us) service_pool.cc:224] Handling call
0709 12:14:12.298397 (+ 14485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.030208 (+731811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.030250 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.043056 (+ 12806us) log.cc:844] Serialized 26 byte log entry
0709 12:14:13.124348 (+ 81292us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:13.124387 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.812889 (+3688502us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540037ee54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:16.812937 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.812977 (+    40us) raft_consensus.cc:1685] finished
0709 12:14:16.812989 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.813200 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152107,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":69,"spinlock_wait_cycles":1221444864,"thread_start_us":260276,"threads_started":2,"wal-append.queue_time_us":307249,"wal-append.run_cpu_time_us":3046,"wal-append.run_wall_time_us":1587791}
I20260709 12:14:16.849658 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:16.742899 26730 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:16.860558 26907 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:16.861020 26907 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:16.862293 23854 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:16.864180 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:16.864821 23859 raft_consensus.cc:2376] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7. Re-sending same reply.
I20260709 12:14:16.861397 26912 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:16.861914 26697 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:14:16.874914 26657 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.876132 26813 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:16.864771 26714 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:16.877480 26657 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.877923 26930 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:16.872612 26711 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:16.867861 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:17.097493 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 235156us
I20260709 12:14:17.099380 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:17.101548 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:17.103211 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:17.085114 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.836404 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:17.104503 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:17.109167 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.110204 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.881516 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2315) took 3721 ms. Trace:
I20260709 12:14:16.942718 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.113232 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2572) took 3094 ms. Trace:
I20260709 12:14:17.113730 23858 rpcz_store.cc:276] 0709 12:14:14.018220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.094818 (+ 76598us) service_pool.cc:224] Handling call
0709 12:14:14.095245 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.095546 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:14.115435 (+ 19889us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.868629 (+2753194us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b180026b780. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:17.112385 (+243756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.112491 (+   106us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:17.112623 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.112753 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.112964 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19604,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":3283,"spinlock_wait_cycles":6540032,"thread_start_us":243055,"threads_started":1,"wal-append.queue_time_us":2868924}
W20260709 12:14:16.882849 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 54900.5us
I20260709 12:14:17.124220 23780 rpcz_store.cc:276] 0709 12:14:13.159874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.555303 (+395429us) service_pool.cc:224] Handling call
0709 12:14:13.573333 (+ 18030us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:15.730999 (+2157666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.881232 (+1150233us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54000bd454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:16.881427 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4932582144}
I20260709 12:14:17.125653 23898 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:16.908931 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 77416.1us
I20260709 12:14:16.881340 23926 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:17.127964 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 282) took 1146 ms. Trace:
I20260709 12:14:17.128378 23926 rpcz_store.cc:276] 0709 12:14:15.980933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.003440 (+ 22507us) service_pool.cc:224] Handling call
0709 12:14:16.802788 (+799348us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:14:17.127667 (+324879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5344640}
I20260709 12:14:17.130577 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.113665 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1031) took 1287 ms. Trace:
I20260709 12:14:16.897243 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.945897 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.135952 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.900269 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:16.949950 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:16.917606 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.142365 23928 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:17.143682 23932 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:17.143997 23931 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:17.148803 23930 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:17.149564 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.150008 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.133306 23856 rpcz_store.cc:276] 0709 12:14:15.826132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.864662 (+ 38530us) service_pool.cc:224] Handling call
0709 12:14:16.942508 (+1077846us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b58001a0a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:14:17.113398 (+170890us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2825472}
I20260709 12:14:17.144994 23927 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:17.152086 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.152808 23927 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:17.153774 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.155071 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.145680 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.156191 23928 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:17.150856 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.145992 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.147099 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:17.145674 23852 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:17.147689 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.159987 23855 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.160614 23934 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:17.161027 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.162349 23858 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:17.163978 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.166127 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:16.995952 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 402344us
I20260709 12:14:17.002103 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 469) took 1103 ms. Trace:
I20260709 12:14:17.169166 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:16.921265 26697 consensus_queue.cc:1243] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:16.922398 26788 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.172267 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:17.174561 26803 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.170303 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.195963 26803 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.923954 26730 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.274s
I20260709 12:14:17.198764 26730 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:17.199530 26919 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:17.200469 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.200752 26788 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.006312 26842 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.175840 26749 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:17.175251 26835 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:16.879621 26733 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.176491 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.177333 26858 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:16.878464 26759 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:16.863695 26907 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:17.178241 26657 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:16.938339 26718 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:16.974918 26789 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:16.998903 26811 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:17.179644 26736 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:17.179939 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.190721 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.198583 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:16.878816 26711 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:17.171027 26716 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.176721 26771 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:17.202894 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.199092 26803 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.287468 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.426760 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.175298 26748 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:17.432709 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.505184 23927 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:17.497432 23860 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:17.498621 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.499457 23852 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:17.509831 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.510500 23858 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:17.510897 23934 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:17.002106 26823 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:17.216331 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.213114 25171 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:17.020044 25119 connection.cc:256] RPC call timeout handler was delayed by -2.9716777549999733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
I20260709 12:14:17.217697 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.515774 23855 raft_consensus.cc:2410] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:14:17.309546 23749 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.523s (SENT) [suppressed 11 similar messages]
I20260709 12:14:17.322501 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:17.521703 26919 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:17.522392 26919 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.213122 26842 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.494432 26657 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.528261 26657 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.528055 26733 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.529399 23930 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:17.529793 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.530658 25176 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.532379 23857 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:17.529141 26901 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:17.533922 26901 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:17.534266 26759 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.542819 23932 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.199141 26730 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:17.504717 26748 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.308391 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.535286 26901 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.493692 26858 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:17.493323 26771 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:16.873310 26912 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.215644 26749 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:17.556849 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.542098 26733 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:17.556000 26771 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:16.984252 26870 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.561074 23854 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:17.218312 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.548866 26748 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.567950 26912 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.558059 26749 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.005411 26804 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.569404 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 350297us
I20260709 12:14:16.892072 26861 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.554044 26858 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.575650 23934 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:17.541955 23680 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.561553 26736 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:17.571318 25176 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:17.579279 25176 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:17.578122 23853 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:17.582137 26711 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.583360 26711 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.585602 26789 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.589269 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.595749 26718 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:17.597242 23858 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:17.597069 26718 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.598713 26718 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.599907 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.600783 23852 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:17.603811 25176 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.605269 25176 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.604300 26730 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.607160 26730 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.607787 26823 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:17.608496 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:17.609764 26733 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:17.609588 26859 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:17.610159 26733 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.610553 26733 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:17.609494 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 53850.7us
I20260709 12:14:17.612684 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.616667 23859 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.612802 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:17.616468 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 302273us
I20260709 12:14:16.917357 26923 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.629374 25171 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.630525 26901 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:17.577004 26811 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.631975 25175 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:17.632005 25119 connection.cc:256] RPC call timeout handler was delayed by -2.2291756620002161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
I20260709 12:14:16.894797 26925 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.617129 23680 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 7, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 7 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:17.641073 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 460031us
W20260709 12:14:17.654570 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.657742 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.663447 25119 leader_election.cc:400] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4.
I20260709 12:14:17.664108 25119 leader_election.cc:403] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:14:17.672137 26639 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 3. Current term is 4.
W20260709 12:14:17.674023 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.678920 26771 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.680823 26592 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:17.685051 23925 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:17.687661 26771 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.694238 26306 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:17.696976 26306 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:17.698693 26771 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.698835 26736 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.699608 26736 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.699520 26771 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:17.702353 26736 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:17.702895 26736 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:17.704366 26592 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:17.705231 26811 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.706395 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50066.6us
I20260709 12:14:17.705910 26572 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:17.719528 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62468.4us
I20260709 12:14:17.703449 26736 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.703356 26771 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:17.734705 26811 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:17.735647 26641 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:17.736166 26641 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:17.736565 26641 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.737347 26923 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.740844 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:17.182902 25172 rpcz_store.cc:276] 0709 12:14:15.898489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.905942 (+  7453us) service_pool.cc:224] Handling call
0709 12:14:15.908152 (+  2210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.908416 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:16.392787 (+484371us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.001414 (+608627us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.001652 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.001707 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:17.001771 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.001847 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.001988 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":405139,"prepare.queue_time_us":12312,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":3480,"raft.queue_time_us":403823,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":676,"thread_start_us":597597,"threads_started":2,"wal-append.queue_time_us":177785}
I20260709 12:14:17.749941 26919 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.755205 26641 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.755892 23851 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:17.757606 26641 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:17.758072 26925 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:17.754505 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 969) took 1704 ms. Trace:
I20260709 12:14:17.766278 25176 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:14:17.766810 25176 pending_rounds.cc:85] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 6
I20260709 12:14:17.759663 26657 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.770283 26731 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.771095 26731 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.726174 26572 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:17.772692 26572 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.773626 26731 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.760876 23925 rpcz_store.cc:276] 0709 12:14:16.049593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.715936 (+666343us) service_pool.cc:224] Handling call
0709 12:14:16.717247 (+  1311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.685652 (+968405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.754164 (+ 68512us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400244ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:17.754421 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171206528}
W20260709 12:14:17.776799 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 267907us
I20260709 12:14:17.773986 26572 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:17.780735 25114 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:17.781662 26592 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:17.783816 26641 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:17.784394 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.785043 23854 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:17.785852 26823 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.786253 26823 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:17.786584 26930 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:17.767104 25176 pending_rounds.cc:107] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:14:17.787906 25176 raft_consensus.cc:2889] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:17.757567 26919 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.792146 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.792471 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:17.806506 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.809047 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.810567 26869 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.811456 26761 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:17.787026 26930 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:17.812561 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3894889480000074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
I20260709 12:14:17.789736 23680 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, term changed from 2 to 3, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to <none>, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:17.793972 26788 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.003s
W20260709 12:14:17.817505 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3862045680000392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30750000000000005s for a response
I20260709 12:14:17.818580 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.821274 23854 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:17.823880 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:17.826612 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.811934 26714 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.787580 26771 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:14:17.835968 26714 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.839107 26716 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:17.839745 26925 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.811951 26761 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.840248 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.839661 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.841413 23859 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:17.829212 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:17.841006 23858 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:17.829394 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.811295 26869 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.840762 26761 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.840291 26925 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.885480 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.886909 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.887185 26925 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:17.847256 23932 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:17.887995 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.847884 23925 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:17.888760 23853 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:17.890204 23933 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
W20260709 12:14:17.851466 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83783.1us
I20260709 12:14:17.891152 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.890414 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:17.891876 23855 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
W20260709 12:14:17.892158 23933 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 964) took 1844 ms (client timeout 1857 ms). Trace:
I20260709 12:14:17.892823 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:17.893595 23855 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.894009 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.895473 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:17.896174 23853 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.884814 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.897228 23785 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:17.897746 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 71907.9us
I20260709 12:14:17.897899 26731 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:17.898773 26731 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:17.899116 26731 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:17.892550 23784 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:17.900110 23928 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.901684 26761 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.858873 26736 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.860088 26947 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:14:17.861979 23681 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 272) took 18789 ms (client timeout 15000 ms). Trace:
I20260709 12:14:17.917387 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:17.919159 23853 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.919734 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.919935 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:17.866400 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:17.920570 25175 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:17.920672 25176 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
W20260709 12:14:17.920809 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54570.4us
I20260709 12:14:17.921531 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:17.921905 26641 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:17.922286 26641 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.922437 26899 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:17.922955 26899 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:17.923765 26899 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.867056 23680 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 6 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.871171 26639 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:17.924141 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.924868 23783 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:17.875582 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.925777 23776 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.926667 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.926870 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.893532 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:17.927402 23784 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:17.870476 26930 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.928254 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:17.880220 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:17.929085 26930 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.880966 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.881088 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.929791 23858 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:17.929296 23680 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.930068 26870 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:17.930842 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.882015 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:17.922910 26641 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.882979 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:17.883356 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.932301 23777 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:17.877422 26714 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.931584 23858 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:17.883879 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.935956 23781 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:17.882524 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:17.937006 23859 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 12:14:17.937281 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 43842.6us
I20260709 12:14:17.913390 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:17.892616 23933 rpcz_store.cc:269] 0709 12:14:16.048209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.368593 (+320384us) service_pool.cc:224] Handling call
0709 12:14:17.891946 (+1523353us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400334ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:17.892079 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3306288384}
I20260709 12:14:17.939180 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:17.915184 23934 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:17.915959 26823 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.852928 26842 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.959072 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.960361 26912 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:17.917898 23681 rpcz_store.cc:269] 0709 12:13:59.072640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.074133 (+  1493us) service_pool.cc:224] Handling call
0709 12:14:17.861634 (+18787501us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:10.194057 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:10.194629 (+   572us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:10.194665 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:14:10.194683 (+    18us) tablet.cc:662] Decoding operations
0709 12:14:10.195407 (+   724us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:10.195463 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:10.195492 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:14:10.196088 (+   596us) tablet.cc:701] Row locks acquired
0709 12:14:10.196108 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:14:10.196274 (+   166us) write_op.cc:270] Start()
0709 12:14:10.196393 (+   119us) write_op.cc:276] Timestamp: P: 1783599250196220 usec, L: 0
0709 12:14:10.196413 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:14:10.197162 (+   749us) log.cc:844] Serialized 2054 byte log entry
0709 12:14:10.973975 (+776813us) op_driver.cc:464] REPLICATION: finished
0709 12:14:11.114435 (+140460us) write_op.cc:301] APPLY: starting
0709 12:14:11.114568 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:11.114880 (+   312us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:11.115136 (+   256us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:11.116407 (+  1271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:11.119949 (+  3542us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:14:11.120024 (+    75us) write_op.cc:312] APPLY: finished
0709 12:14:11.120400 (+   376us) log.cc:844] Serialized 47 byte log entry
0709 12:14:11.120650 (+   250us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:11.120806 (+   156us) write_op.cc:454] Released schema lock
0709 12:14:11.120866 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:11.188861 (+ 67995us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:11.190046 (+  1185us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2639815,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":136793,"apply.run_cpu_time_us":3790,"apply.run_wall_time_us":22867,"num_ops":5,"prepare.queue_time_us":111754,"prepare.run_cpu_time_us":28838,"prepare.run_wall_time_us":146664,"raft.queue_time_us":218726,"raft.run_cpu_time_us":4612,"raft.run_wall_time_us":155916,"replication_time_us":733921,"thread_start_us":356630,"threads_started":4,"wal-append.queue_time_us":202832,"wal-append.run_cpu_time_us":21674,"wal-append.run_wall_time_us":1762674}]]}
I20260709 12:14:17.960327 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.962576 26912 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.963202 23776 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:17.963963 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 242067us
I20260709 12:14:17.893146 26592 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:14:17.927192 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:17.931565 26870 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:17.903365 23854 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:17.929513 23780 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.939865 26639 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:17.940027 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.904834 26869 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:17.941216 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.931525 23680 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.943298 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.914472 26731 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:17.905707 26733 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:17.946305 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46631us
W20260709 12:14:17.909197 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.920166 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:17.865010 26657 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:17.987068 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.988046 23853 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:17.966940 23777 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:17.989626 23680 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 5 to 7, term changed from 6 to 7, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to <none>, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 7 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:17.964291 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66722.8us
W20260709 12:14:17.970490 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.972690 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.970283 23782 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:14:17.983700 23781 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:17.979413 23784 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:17.990479 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:17.978214 26639 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.970672 26870 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.958192 26714 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:14:17.987730 23785 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4.
I20260709 12:14:17.971697 26858 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.991218 23900 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.005160 23776 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:18.006305 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.006424 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.008656 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.007289 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:17.968317 26748 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.978483 26749 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:17.993873 23680 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 8, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.997274 26639 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.010372 23783 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:18.015630 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.002305 26870 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.018008 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.029872 23780 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:18.041719 23781 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:18.043948 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:18.042732 23680 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:18.048903 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:18.049348 26749 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:18.049582 26823 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.049809 23853 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:18.050180 26955 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:18.050630 26955 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:18.051012 26955 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.051415 26842 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.055749 26826 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:18.056293 26826 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:18.057451 26826 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.058826 26826 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.059801 26858 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:18.059847 26592 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:18.060000 23778 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.005090 26907 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.029927 26788 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:18.071164 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:18.072315 23852 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:18.063166 23680 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 4 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:18.082835 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 142174us
I20260709 12:14:18.087456 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:18.097947 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:18.063879 26592 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:18.071107 26731 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:18.071542 26788 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.105755 23782 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:14:18.109216 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:18.113793 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 370821us
I20260709 12:14:18.115617 23785 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:18.131311 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83528.3us
W20260709 12:14:18.126032 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.142748 23853 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.132381 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.147186 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.147811 23778 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.123485 26641 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.152470 26641 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.147207 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.156473 23780 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:18.171705 23779 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:18.176607 23853 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:18.143664 26788 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.181577 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.182554 26831 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:18.183063 26831 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:18.133164 26592 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:18.194507 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.195479 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.196081 26761 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:18.131837 26925 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.199954 23785 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:18.201385 23780 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:18.127451 26748 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:18.203685 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 264543us
I20260709 12:14:18.213590 23779 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.215068 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.126451 26899 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:18.218082 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.215196 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.228129 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.229003 23782 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:14:18.234159 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:18.235273 26930 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:18.236078 26930 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.238382 26930 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:18.240645 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:18.235273 23785 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:18.242918 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:18.243229 26733 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.245003 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:18.245255 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.245679 23859 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.247957 26718 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:18.248459 26718 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.248795 26718 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:18.248845 26907 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:18.251186 26714 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:18.251899 26714 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.270838 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.271828 23779 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.273535 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.263696 26641 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:14:18.276486 26962 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:18.283949 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:18.284809 25175 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:18.285925 26788 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.286749 26714 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.288591 26788 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:18.289330 23859 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:18.290016 23778 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 4: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.291956 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:18.292802 23858 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:18.290650 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.295271 23785 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:18.304255 26639 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.307535 26639 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:18.305284 26641 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:18.307879 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.307462 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.309937 23781 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:18.308235 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.313627 23778 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.251371 26831 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.321568 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.322525 23781 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:14:18.346513 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.309670 26714 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:18.313932 26641 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:18.309398 26962 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:14:18.355640 26641 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:18.390625 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.391947 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.392614 23785 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:18.391358 23779 raft_consensus.cc:2376] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 4. Re-sending same reply.
I20260709 12:14:18.366098 26714 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:18.401456 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.355134 26955 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.406402 23785 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:18.394290 26962 consensus_queue.cc:1243] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:14:18.421053 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.426024 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 107983us
I20260709 12:14:18.382620 26870 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.432686 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.450069 23778 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:18.450826 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:18.439792 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 166310us
I20260709 12:14:18.416045 26761 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:18.439400 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.454595 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:18.463574 23852 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:18.422982 26899 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:18.466997 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.467694 23778 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:18.410267 26963 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:14:18.433846 26962 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:18.477459 26962 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:18.492429 26961 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:14:18.549541 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73332.9us
I20260709 12:14:18.465907 25172 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:18.534657 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83952.6us
I20260709 12:14:18.568742 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:18.570116 23852 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.553275 26811 raft_consensus.cc:2764] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:18.609834 25171 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:18.619390 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.646027 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.548659 26761 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.650164 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:18.656195 26964 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:14:18.656629 26730 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:18.657301 26730 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:18.668262 26730 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.669193 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.548420 26641 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.558691 26714 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.787606 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2608) took 1159 ms. Trace:
I20260709 12:14:18.660037 26979 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:18.809513 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 330750us
I20260709 12:14:18.812354 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:18.788096 23856 rpcz_store.cc:276] 0709 12:14:17.595551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.598210 (+  2659us) service_pool.cc:224] Handling call
0709 12:14:17.598559 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.598903 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:17.827179 (+228276us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.314293 (+487114us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.754756 (+440463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:18.754817 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:18.754875 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.754943 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.755082 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71366,"prepare.queue_time_us":419961,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":479324,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"thread_start_us":937793,"threads_started":3,"wal-append.queue_time_us":543937}
W20260709 12:14:18.768210 26731 consensus_meta.cc:287] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.617s	user 0.007s	sys 0.003s
I20260709 12:14:18.829080 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.830103 23778 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:18.817808 23776 raft_consensus.cc:2376] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8. Re-sending same reply.
W20260709 12:14:18.825311 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.835244 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:18.835808 26966 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:14:18.819103 26869 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:14:18.837244 26869 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:18.837633 26869 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:18.836251 23858 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:18.842742 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.813437 26979 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.659541 26962 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.848665 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.855423 23778 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:18.839821 26955 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:18.838742 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:18.861155 23776 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:18.847525 26641 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.866613 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:18.868538 25176 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.869071 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:18.873831 23852 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:18.851316 26979 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.859282 26919 raft_consensus.cc:2764] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:18.864024 26870 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:18.884183 26965 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 12:14:18.884549 26592 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
W20260709 12:14:18.884905 26969 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:14:18.876782 25168 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:18.882201 26718 consensus_meta.cc:287] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.004s
I20260709 12:14:18.908542 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:18.953958 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 284832us
I20260709 12:14:18.953233 25175 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:14:18.962438 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 53893.2us
I20260709 12:14:18.906234 26981 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.909245 26718 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:18.898057 26592 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.867090 26869 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:18.965757 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2354) took 1021 ms. Trace:
W20260709 12:14:19.001211 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 299890us
W20260709 12:14:18.977946 23785 consensus_meta.cc:287] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.000s
I20260709 12:14:19.002061 23785 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:19.000195 23777 rpcz_store.cc:276] 0709 12:14:17.944672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.018988 (+ 74316us) service_pool.cc:224] Handling call
0709 12:14:18.019472 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.019844 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:14:18.019877 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.020497 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.315450 (+294953us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:18.315496 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.315567 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:18.965500 (+649933us) raft_consensus.cc:1685] finished
0709 12:14:18.965525 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.965676 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":293238,"threads_started":1,"wal-append.queue_time_us":386612}
W20260709 12:14:19.004379 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.011338 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.013332 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.994689 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:19.018492 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.021003 25175 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:14:18.998889 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2355) took 1052 ms. Trace:
W20260709 12:14:19.032271 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.989790 25176 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:18.964519 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:18.980492 26718 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:19.050439 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 395607us
I20260709 12:14:19.051987 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:19.053023 26733 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:18.970541 26761 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:19.037117 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.734s (SENT)
I20260709 12:14:19.067134 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:19.060057 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.066926 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:19.062331 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 52220.7us
I20260709 12:14:18.977519 26592 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:19.053602 26733 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:19.019881 25169 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:19.054857 26869 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:19.081453 26973 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.366s	user 0.002s	sys 0.000s
I20260709 12:14:18.966251 26981 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.069330 26979 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.105211 26979 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:19.106248 26831 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:19.107321 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.108310 25173 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:19.090965 26982 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:14:19.092991 26955 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:19.111145 26955 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:19.115903 23853 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:19.117395 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:19.094892 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 130428us
I20260709 12:14:19.101225 23785 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:19.101778 26987 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:19.117981 23853 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:19.108600 26733 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.118743 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:19.023138 23783 rpcz_store.cc:276] 0709 12:14:17.947331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.028752 (+ 81421us) service_pool.cc:224] Handling call
0709 12:14:18.071669 (+ 42917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.144695 (+ 73026us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:18.422962 (+278267us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.867564 (+444602us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.995395 (+127831us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:18.995458 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:18.995915 (+   457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.996603 (+   688us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.997725 (+  1122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":405515,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":76,"raft.queue_time_us":28718,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":11110,"spinlock_wait_cycles":19162496,"thread_start_us":785207,"threads_started":3}
W20260709 12:14:19.120880 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 154383us
I20260709 12:14:19.103237 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.132715 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:19.133481 23856 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.106133 26967 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:19.137987 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:19.166020 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:19.175928 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 47360.4us
I20260709 12:14:19.177321 23853 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:19.121441 26987 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:19.192950 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.105245 26981 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:19.224582 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:19.225054 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.225310 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48594.9us
I20260709 12:14:19.138455 26967 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:19.164191 26870 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:19.137959 26641 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:14:19.217526 26718 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.203504 26987 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.226708 26995 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:19.168313 27003 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 12:14:19.239318 26981 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:19.296504 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.300783 23927 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:14:19.299042 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 283959us
I20260709 12:14:19.262814 26870 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.309319 26955 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.309466 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.308792 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.226850 26993 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:14:19.367172 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.367540 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.355620 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:19.368418 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.357489 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:19.365890 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:19.369220 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.391413 23927 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:19.389524 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 354) took 1117 ms. Trace:
I20260709 12:14:19.393513 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:19.394469 23933 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:19.335561 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.314766 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.396425 23928 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.397330 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.390146 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 356) took 1087 ms. Trace:
I20260709 12:14:19.398731 23934 raft_consensus.cc:2410] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:14:19.391388 23851 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:19.397104 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 350) took 1180 ms. Trace:
I20260709 12:14:19.400676 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:19.402827 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 103130us
I20260709 12:14:19.398224 23858 rpcz_store.cc:276] 0709 12:14:18.302557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.853370 (+550813us) service_pool.cc:224] Handling call
0709 12:14:18.862177 (+  8807us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.862442 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:14:18.862475 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.875233 (+ 12758us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.127750 (+252517us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:19.127793 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.127857 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.189208 (+ 61351us) raft_consensus.cc:1685] finished
0709 12:14:19.189234 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.390083 (+200849us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":174277,"threads_started":1,"wal-append.queue_time_us":275655}
I20260709 12:14:19.405727 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:19.406656 26639 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:19.407208 26639 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:19.407559 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.408149 26992 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.408149 26639 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.408711 26992 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.311048 26870 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:19.412287 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.401479 26788 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:19.413887 26788 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.417189 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.417899 23927 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:19.392102 23929 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:19.311359 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.420938 23930 raft_consensus.cc:2410] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:14:19.393206 23859 rpcz_store.cc:276] 0709 12:14:18.272469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.290065 (+ 17596us) service_pool.cc:224] Handling call
0709 12:14:18.311422 (+ 21357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.586063 (+274641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:18.586098 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.621587 (+ 35489us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.024773 (+403186us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:19.024823 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.024904 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.389280 (+364376us) raft_consensus.cc:1685] finished
0709 12:14:19.389305 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.389459 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":500288,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8886,"thread_start_us":602436,"threads_started":2,"wal-append.queue_time_us":525742}
I20260709 12:14:19.423362 23851 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:19.423458 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:19.296888 26995 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.431051 23933 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:19.452975 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.453902 23933 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.454082 26831 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.429401 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:19.456815 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.273272 26718 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.401671 23857 rpcz_store.cc:276] 0709 12:14:18.216963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.217073 (+   110us) service_pool.cc:224] Handling call
0709 12:14:18.275422 (+ 58349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.609644 (+334222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:18.609676 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.610244 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.998771 (+388527us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:18.998815 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.998890 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.312842 (+313952us) raft_consensus.cc:1685] finished
0709 12:14:19.312865 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.397041 (+ 84176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":351282,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":51171,"thread_start_us":657193,"threads_started":2,"wal-append.queue_time_us":570371}
W20260709 12:14:19.463076 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.468638 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2376) took 1043 ms. Trace:
I20260709 12:14:19.463726 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:19.468163 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.469076 23781 rpcz_store.cc:276] 0709 12:14:18.425435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.404302 (+978867us) service_pool.cc:224] Handling call
0709 12:14:19.468572 (+ 64270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:19.470778 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.478276 23931 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:19.479146 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 351053us
I20260709 12:14:19.454696 26831 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:19.484650 23749 connection.cc:256] RPC call timeout handler was delayed by -2.7894120980001844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2747s for a response
W20260709 12:14:19.487756 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 83674.7us
I20260709 12:14:19.489719 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:19.491912 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.456509 26995 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.503551 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.503813 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.464803 26919 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:19.515432 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.513957 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2358) took 1492 ms. Trace:
I20260709 12:14:19.480351 26826 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.517745 23784 rpcz_store.cc:276] 0709 12:14:18.014204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.029313 (+ 15109us) service_pool.cc:224] Handling call
0709 12:14:18.044984 (+ 15671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.060706 (+ 15722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:18.060739 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.087326 (+ 26587us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.368162 (+280836us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:18.368254 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.368344 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.403283 (+1034939us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b240018aa18. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa54b9c4 00007fb2aa549acb 00007fb2aa704dc9 00007fb2aa704a55 00007fb2aa703fb0 00007fb2ad37a488 00007fb2ad37abe8 00007fb2ad352398 00007fb2ad34d76a 00007fb2ad34cbb6
0709 12:14:19.496642 (+ 93359us) raft_consensus.cc:1685] finished
0709 12:14:19.496668 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.503716 (+  7048us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16459,"raft.queue_time_us":3708,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13441,"spinlock_wait_cycles":85530880,"thread_start_us":247587,"threads_started":1,"wal-append.queue_time_us":453234}
W20260709 12:14:19.543048 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.549566 23824 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:19.516085 26975 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:19.550697 23824 leader_election.cc:400] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:14:19.533881 27021 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:19.545056 26731 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.559367 23824 leader_election.cc:403] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:14:19.559577 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.585139 23856 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:19.587651 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 85297.9us
I20260709 12:14:19.560281 26979 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:19.545193 26826 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.571483 25170 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:19.575814 27021 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:19.585682 26731 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.592149 25172 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
W20260709 12:14:19.619102 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.613289 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.635398 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.626579 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:19.663801 23749 connection.cc:256] RPC call timeout handler was delayed by -1.882811047000132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 12:14:19.627291 23928 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:19.664319 23749 connection.cc:256] RPC call timeout handler was delayed by -1.7937121810000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.48910000000000003s for a response
W20260709 12:14:19.632815 23825 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.533s (SENT)
W20260709 12:14:19.664856 23749 connection.cc:256] RPC call timeout handler was delayed by -1.3640254849999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 12:14:19.588202 26981 raft_consensus.cc:2764] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:14:19.665261 23749 connection.cc:256] RPC call timeout handler was delayed by -1.2038254849999248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 12:14:19.666383 23749 connection.cc:256] RPC call timeout handler was delayed by -1.1498254850000649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:14:19.665125 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:19.590998 26979 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:19.667322 23933 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:19.663722 23930 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:19.598963 26919 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6.
I20260709 12:14:19.607805 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:19.615839 27019 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:14:19.619458 26869 consensus_meta.cc:287] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
I20260709 12:14:19.611344 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.674993 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.613116 26826 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:19.598480 25170 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:19.676625 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.677034 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44268.8us
I20260709 12:14:19.675323 26962 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.672649 26979 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:19.608204 27021 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.672164 26993 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:19.676319 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.628933 26831 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:19.717772 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.718622 25171 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:19.681097 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:19.719985 23926 raft_consensus.cc:2410] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:19.610056 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.685664 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.685959 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.686954 23783 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:19.685689 26992 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:19.687609 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.688200 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:19.688629 27023 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:14:19.689435 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.689695 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.740336 23927 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:19.740595 23781 raft_consensus.cc:2410] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:19.689466 23784 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:14:19.618032 25168 consensus_meta.cc:287] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.002s
I20260709 12:14:19.699460 25174 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:14:19.741168 23825 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.555s (SENT)
I20260709 12:14:19.677950 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.712030 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.742210 23933 raft_consensus.cc:2410] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:14:19.672133 27022 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:19.715097 23931 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.716063 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.716773 23858 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:19.721696 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.722935 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.738381 23776 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:19.739915 23777 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.743194 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:19.742211 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.567s (SENT)
I20260709 12:14:19.744963 25168 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:19.745500 26731 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:19.748664 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:19.701601 26975 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:19.750141 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.880558 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:19.880779 27020 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:14:19.872324 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.634056 26967 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:19.879083 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.876060 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.872218 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.889241 23782 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:19.886966 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 144780us
I20260709 12:14:19.883924 23781 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:19.884706 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:19.891017 23784 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:19.891196 23926 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:19.892729 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.893128 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:19.751946 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:19.854338 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:19.760586 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.887943 23934 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:19.753327 23778 raft_consensus.cc:2376] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10. Re-sending same reply.
I20260709 12:14:19.901048 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:19.905031 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 161895us
I20260709 12:14:19.909345 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.907178 23778 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:19.912863 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 242375us
I20260709 12:14:19.786015 27021 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.740072 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:19.921486 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.906565 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.922503 23781 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:19.924242 25174 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:19.727298 26718 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:19.924832 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:19.926565 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.925755 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.926762 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.913952 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:19.890477 26963 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:19.911808 25168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 720) took 1426 ms (client timeout 1761 ms). Trace:
I20260709 12:14:19.929867 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.929008 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.942152 23776 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:19.932839 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:19.908159 26788 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:19.947729 23778 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:19.948817 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.896024 27022 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:19.954196 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 403664us
I20260709 12:14:19.975904 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.981521 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:19.954623 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.979349 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:19.967859 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.891394 26985 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:14:19.984617 23931 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:19.986455 23777 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:19.987521 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.922739 26592 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:19.980391 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83437.7us
I20260709 12:14:19.974320 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:19.990947 23778 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.978299 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:19.997969 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 104913us
I20260709 12:14:19.998749 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:20.001547 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:20.000481 23751 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.762s (SENT)
I20260709 12:14:19.947944 26748 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:20.002663 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:20.007493 23780 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:20.002784 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 118020us
I20260709 12:14:19.996649 25173 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.002018 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:19.945698 26714 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:20.004065 23858 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:20.005466 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:19.973174 26987 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.659s	user 0.005s	sys 0.005s
I20260709 12:14:20.005920 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:19.969228 27027 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:14:19.996758 26831 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.012127 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:19.946527 26869 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:14:20.049824 23784 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:20.013468 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:20.016821 23856 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.018568 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:19.941537 25168 rpcz_store.cc:269] 0709 12:14:18.468720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.470516 (+  1796us) service_pool.cc:224] Handling call
0709 12:14:19.894936 (+1424420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:20.019408 23857 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:20.020172 26718 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:14:20.021267 27012 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.329s	user 0.000s	sys 0.004s
I20260709 12:14:20.020627 27029 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:14:20.021355 23785 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:14:20.018898 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:20.065205 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46381.3us
I20260709 12:14:20.021883 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:20.016213 26714 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:20.022651 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.913s (SENT)
I20260709 12:14:20.069782 23929 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:20.033109 26306 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:20.034121 26987 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:19.927388 25170 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.088979 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:20.035857 26990 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:20.038276 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.039772 26975 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.013337 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:20.051155 23931 raft_consensus.cc:2368] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:19.951102 26639 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:20.020444 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:20.055378 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.056193 23777 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:20.060598 23779 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:14:20.061919 23776 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:20.011152 26592 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:20.065853 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:20.095414 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.098623 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 199717us
W20260709 12:14:20.099345 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.616s (SENT)
I20260709 12:14:20.072525 26714 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.101943 23781 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:20.072558 26831 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:20.103267 23776 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:20.099836 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:20.104079 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:20.110658 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88088.1us
W20260709 12:14:20.118952 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 117008us
I20260709 12:14:20.100353 26990 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:20.104511 25170 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:20.121855 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:20.104446 26730 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:20.120397 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.106254 26831 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:20.142097 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 244009us
I20260709 12:14:20.134624 26731 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:20.108014 27003 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 12:14:20.342304 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 148200us
W20260709 12:14:20.326905 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 261361us
I20260709 12:14:20.320312 23859 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:14:20.326689 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 204931us
I20260709 12:14:20.342206 23778 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:20.328594 23781 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:14:20.352922 23899 leader_election.cc:336] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.155s (SENT)
W20260709 12:14:20.357826 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 258051us
I20260709 12:14:20.317873 23858 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:20.266880 26993 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:20.370673 26993 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:20.371726 26993 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.374308 26993 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:20.372686 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:20.373610 25170 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:14:20.378162 25170 pending_rounds.cc:85] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 6
I20260709 12:14:20.376603 26788 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:20.379077 25170 pending_rounds.cc:107] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:14:20.379452 25170 raft_consensus.cc:2889] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:20.380525 26899 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:14:20.367244 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:20.383317 23899 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.557s (SENT)
I20260709 12:14:20.383824 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:20.384621 27050 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:20.386092 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.388124 27050 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.388590 27050 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.316958 23782 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:20.385393 27003 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:20.177964 26592 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.156167 26831 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.389048 27050 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.168568 27016 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:20.272996 26987 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:20.156309 26826 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.394768 26826 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.395741 26748 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:20.396332 26572 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:20.396888 26572 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:20.397447 27034 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:20.176550 27032 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:14:20.203789 25171 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:20.273741 26919 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.397794 26826 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.411870 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.412824 23859 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:20.413705 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.414937 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.409101 23851 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:20.415896 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.415395 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.417790 23855 raft_consensus.cc:2410] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:20.417821 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.409025 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.391158 26714 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:20.263329 26718 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:20.415496 23859 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.409709 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.428684 23852 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.418812 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.419693 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.421592 26967 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.422171 26733 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:20.422854 26761 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:20.424669 23857 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:20.445794 23930 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:20.445900 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:20.446677 25170 raft_consensus.cc:2376] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:14:20.446907 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.447522 23931 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:20.447702 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:20.448570 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.448351 25168 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:20.449332 23933 raft_consensus.cc:2410] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:20.450237 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.450845 23929 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:20.453409 23934 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:20.453017 27050 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.273558 27022 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:20.455931 26718 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.426241 23858 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:20.460541 26899 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:20.398707 27016 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:20.461473 27016 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.464998 26985 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:20.431358 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.397851 26572 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.429749 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.427515 27034 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:20.438294 23851 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:14:20.439779 26788 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:20.470104 23898 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:20.470999 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52257.8us
I20260709 12:14:20.472554 23859 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
W20260709 12:14:20.425331 26979 consensus_meta.cc:287] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.698s	user 0.006s	sys 0.004s
I20260709 12:14:20.502619 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.504611 23780 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:20.442754 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59478.1us
W20260709 12:14:20.426947 26995 consensus_meta.cc:287] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.003s
I20260709 12:14:20.477874 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.482353 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.484855 23855 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.485940 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.513586 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:20.514027 23858 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.480046 27042 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.510347 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:20.516929 23930 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.481182 27034 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.521534 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.525889 23854 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:20.521555 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.526706 23852 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:20.522691 23931 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:20.528255 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 139744us
I20260709 12:14:20.476956 26733 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.518981 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.508298 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.534828 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.532027 23926 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
W20260709 12:14:20.532984 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 146940us
I20260709 12:14:20.478360 26993 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:20.519140 26826 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:20.537021 23857 raft_consensus.cc:2368] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8.
W20260709 12:14:20.537909 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 117617us
I20260709 12:14:20.537534 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.281000 26990 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.535689 23933 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:20.530797 27034 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:20.546064 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:20.539275 23781 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:20.551033 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.561483 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.568022 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.569031 23926 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:20.540870 26731 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.511919 26979 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.586936 23931 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:20.515834 26995 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.545934 26975 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:20.577929 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:20.597234 23933 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:20.579039 26826 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:20.589286 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.588809 26748 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:20.596553 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.604776 26826 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:20.605504 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.605734 26748 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:20.607533 23899 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 3.167s (SENT)
I20260709 12:14:20.608069 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:20.611616 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.615474 25170 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:20.578728 26731 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.619984 26731 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.623023 26748 raft_consensus.cc:2764] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:20.622812 27026 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:20.625205 26995 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.624929 27016 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.626071 23853 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.627482 23776 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:20.628677 27016 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.629326 27016 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:20.630090 27016 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:20.630604 27016 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:20.542862 26733 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.529721 27042 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.634387 26733 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.635659 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:20.642976 26979 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.550732 27039 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 12:14:20.648483 26731 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.650689 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.651516 23855 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:20.652324 23825 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:20.573419 26962 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.572559 27022 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.651506 23856 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 4: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:20.576666 27050 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:20.659734 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 70473.6us
I20260709 12:14:20.667706 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:20.622870 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.654040 26718 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:20.658411 27022 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:20.631414 27016 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:20.662258 26979 raft_consensus.cc:2764] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:14:20.675683 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.678539 26993 raft_consensus.cc:2764] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:20.669826 26962 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:20.692915 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.709894 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 147346us
I20260709 12:14:20.711670 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:20.676474 26718 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:20.717451 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 94648.8us
I20260709 12:14:20.722200 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.681350 27022 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.723667 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:20.726913 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.695698 27050 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:20.747063 27062 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:20.718963 26993 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.700485 25170 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:20.723006 26718 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:20.769158 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.753901 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.755985 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:20.762459 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.727705 26714 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:20.786670 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.792771 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:20.787463 23857 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:20.796458 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.799515 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.753459 27058 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.736996 26831 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:20.802279 26981 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:20.817961 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:20.818907 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.821782 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67931.5us
I20260709 12:14:20.823043 27054 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:20.813571 26748 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:20.826475 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.828197 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.833787 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.835379 23856 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.842000 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:20.841435 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:20.832255 25172 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:20.854753 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:20.856189 23785 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:20.821386 26981 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:20.859333 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:20.859449 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:20.833271 27050 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:20.860256 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.861359 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:20.860342 23780 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:20.862080 23856 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:20.862409 25176 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.861259 26995 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:20.864616 26981 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:20.843223 26731 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:20.839126 27058 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:20.868834 23857 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:14:20.859031 26993 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.907747 25114 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:20.864493 27074 raft_consensus.cc:2764] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:14:20.912173 23930 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:20.861681 27050 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:20.915297 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62936.5us
I20260709 12:14:20.860216 26975 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:14:20.910262 26993 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.909725 27041 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:20.924633 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.926138 25176 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:20.914208 26962 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:20.927348 26993 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.916747 27050 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:20.938433 23857 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
W20260709 12:14:20.942384 27070 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:14:20.921811 27058 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:20.947141 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 92422.4us
I20260709 12:14:20.927205 27041 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.948118 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:20.953476 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0123161609999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 12:14:20.957311 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:20.957402 23852 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:20.957958 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.960736 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:20.961314 23857 raft_consensus.cc:2376] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8. Re-sending same reply.
I20260709 12:14:20.964848 26831 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:20.964854 27063 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:14:20.965634 26592 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:20.967185 27042 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:20.970942 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:20.971736 23852 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:20.951974 27041 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.977613 27022 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:20.949744 27034 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:20.980831 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:21.025899 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68669us
W20260709 12:14:21.011353 23853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 2636) took 1709 ms (client timeout 1932 ms). Trace:
W20260709 12:14:21.024026 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51109.7us
I20260709 12:14:20.959923 27074 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:21.027133 23853 rpcz_store.cc:269] 0709 12:14:19.301785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.304994 (+  3209us) service_pool.cc:224] Handling call
0709 12:14:21.011146 (+1706152us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540037f854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:21.011290 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2832676224}
I20260709 12:14:21.026684 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.030305 27074 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:21.029140 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:21.030752 27074 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.036831 23860 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:21.015743 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:20.984819 23751 leader_election.cc:336] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.599s (SENT)
I20260709 12:14:20.963898 27058 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:21.040015 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.037600 26788 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:21.071460 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 86674.2us
W20260709 12:14:21.072512 23751 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.654s (SENT)
I20260709 12:14:21.009896 27022 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:21.073192 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:21.025396 27080 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.010432 26748 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:21.149161 27058 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.159952 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:21.146885 26748 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:21.214548 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66605.5us
I20260709 12:14:21.066071 27079 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:21.206578 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.265671 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:21.267807 25171 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:21.269387 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 470119us
I20260709 12:14:21.272336 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:21.272904 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:21.273617 23856 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:21.278411 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.045066 23857 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:21.212522 23854 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:21.162480 27022 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.234674 27080 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:21.233937 23825 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.561s (SENT)
W20260709 12:14:21.233767 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 161283us
I20260709 12:14:21.284873 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:21.238981 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.240499 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:21.163053 26718 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.241019 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:21.237232 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 210600us
W20260709 12:14:21.285244 23751 leader_election.cc:336] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.777s (SENT)
I20260709 12:14:21.289860 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.290118 23934 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:21.298779 23824 leader_election.cc:400] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 4 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 4. Current term is 5.
I20260709 12:14:21.282580 26981 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:21.300252 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:21.301072 27083 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 12:14:21.303623 23751 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.837s (SENT)
I20260709 12:14:21.304152 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.306272 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.307210 27079 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:21.308066 27079 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:21.308369 27084 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:14:21.308789 27079 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:21.309396 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2647) took 1300 ms. Trace:
I20260709 12:14:21.309489 27079 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:21.311470 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:21.313423 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.312063 23860 rpcz_store.cc:276] 0709 12:14:20.008876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.009021 (+   145us) service_pool.cc:224] Handling call
0709 12:14:21.011818 (+1002797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.037279 (+ 25461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.309179 (+271900us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540037f850. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:21.309327 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305332608}
W20260709 12:14:21.319914 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 86004.1us
W20260709 12:14:21.320616 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.596s (SENT)
I20260709 12:14:21.321029 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.324201 26979 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.324927 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.325619 23780 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:21.318204 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:21.326793 23854 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:21.319064 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:21.324832 27081 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:14:21.340173 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.623s (SENT)
I20260709 12:14:21.340663 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.342377 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.339643 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.348687 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.303598 26718 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:21.366657 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.367430 23780 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:21.337214 26975 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:21.338410 26869 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:21.371308 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.371865 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.300299 26981 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.385885 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.386436 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:21.386598 23778 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:21.387346 27068 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:21.388371 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.312711 27080 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.392669 23780 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.412642 23780 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:21.369889 27026 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.389086 23783 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:21.414547 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.404748 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.417325 23782 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:21.418272 23780 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:21.419068 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:21.419965 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.420665 23778 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:21.421926 23778 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.422714 23780 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:21.423007 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:21.423877 23784 raft_consensus.cc:2393] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 5: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:21.424811 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:21.392197 26718 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.367471 27034 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:21.434165 23785 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:21.391815 26869 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:21.439157 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:21.431731 27078 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:14:21.443189 23783 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:21.444198 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:21.445277 23852 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:21.452474 26979 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:14:21.456400 27076 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:14:21.459400 26718 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:21.459939 26718 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:21.460853 26718 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.426450 27026 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.435984 26975 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:21.465309 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.466957 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 416) took 1246 ms. Trace:
I20260709 12:14:21.426405 27087 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:21.497696 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 191486us
I20260709 12:14:21.485176 23776 rpcz_store.cc:276] 0709 12:14:20.220960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.221136 (+   176us) service_pool.cc:224] Handling call
0709 12:14:20.272867 (+ 51731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:20.627917 (+355050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.466701 (+838784us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b540010a254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:21.466891 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815095680}
I20260709 12:14:21.490682 23858 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:21.505059 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.508670 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:21.509459 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 169302us
W20260709 12:14:21.511914 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.712s (SENT)
W20260709 12:14:21.501389 27089 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.218s	user 0.002s	sys 0.000s
W20260709 12:14:21.501533 27088 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.263s	user 0.002s	sys 0.000s
W20260709 12:14:21.532644 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.790s (SENT)
I20260709 12:14:21.533108 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:21.499286 26899 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.546819 23852 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:21.554561 26981 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:21.554466 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.966s (SENT)
W20260709 12:14:21.556581 23825 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.974s (SENT)
I20260709 12:14:21.556984 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:21.557301 27074 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.003s
I20260709 12:14:21.557778 27074 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.558279 23858 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:21.558545 26981 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:21.558104 23825 leader_election.cc:336] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.051s (SENT)
W20260709 12:14:21.559680 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.067s (SENT)
I20260709 12:14:21.560137 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:21.559764 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51178us
I20260709 12:14:21.560277 25168 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:21.563370 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 215889us
I20260709 12:14:21.563829 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.567492 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.573603 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.541391 27003 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:14:21.579191 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.365s (SENT)
I20260709 12:14:21.579743 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.566167 25170 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.538528 27090 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:14:21.571719 26981 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:21.550534 26899 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.581135 26975 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:21.588080 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.560932 26869 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.578969 27087 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:21.586781 27101 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:21.588239 27074 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.606668 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1060) took 1182 ms. Trace:
W20260709 12:14:21.608639 25172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 248) took 2033 ms (client timeout 1580 ms). Trace:
W20260709 12:14:21.609103 25172 rpcz_store.cc:269] 0709 12:14:19.575032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.717445 (+142413us) service_pool.cc:224] Handling call
0709 12:14:21.608404 (+1890959us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b540017d4d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:21.608557 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060593536}
I20260709 12:14:21.588176 27097 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:14:21.616611 27050 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.000s
I20260709 12:14:21.617101 27050 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:21.619022 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.599496 27026 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:21.590488 27099 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:14:21.620462 27099 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:21.620813 27099 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:21.621590 27099 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.623018 27099 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:21.621541 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.624068 26899 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:21.625387 26993 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.685s	user 0.008s	sys 0.004s
I20260709 12:14:21.595032 27021 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:21.619315 27050 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.607100 23855 rpcz_store.cc:276] 0709 12:14:20.424273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.424781 (+   508us) service_pool.cc:224] Handling call
0709 12:14:21.606498 (+1181717us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54002f53d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:21.606610 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36479360}
I20260709 12:14:21.636619 23854 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:21.643610 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55719.2us
I20260709 12:14:21.648517 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.635885 27026 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.651248 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:21.653620 23930 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.654690 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.625895 26993 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.639542 27068 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:14:21.648118 27103 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:21.663393 23859 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:21.657945 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.638051 27101 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.661815 27104 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:21.645269 27080 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.669101 27100 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:21.669545 26993 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.669929 26970 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:21.670475 26970 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.672782 26970 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:21.659340 26981 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:21.674001 26981 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.674363 26981 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:21.674556 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:21.680858 26788 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:21.667713 26975 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:21.683234 26975 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.683380 26730 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:21.685746 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.686797 23933 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
W20260709 12:14:21.688045 27041 consensus_meta.cc:287] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.009s
I20260709 12:14:21.688580 27041 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.692553 27111 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.698308 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:21.700012 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:21.700726 25172 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.705017 27101 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.706446 26748 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.708196 23784 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:21.709128 26748 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.663317 27103 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.711376 27103 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:21.711376 23782 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.709313 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.689759 23933 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:21.773312 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.774127 23777 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:21.664878 26869 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:21.812521 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:21.660913 27026 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.812901 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.679941 27104 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:21.817466 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:21.727720 26975 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:21.825232 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 225945us
I20260709 12:14:21.826135 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:21.836191 27104 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.838399 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:21.842679 27003 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.843304 27003 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.846022 27050 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.846873 27041 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:21.693317 27111 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:21.849093 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.854810 23930 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:14:21.857656 23899 leader_election.cc:336] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.583s (SENT)
I20260709 12:14:21.822093 27068 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:21.871096 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:21.815737 26970 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:14:21.888398 26970 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:21.888827 26970 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:21.889703 26970 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.832587 26788 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:21.846902 27050 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:21.877471 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:21.878778 27109 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:14:21.909205 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:21.848383 27022 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:14:21.902884 26970 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:21.921106 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.921654 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.905385 27101 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.903393 27068 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:21.945312 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.945613 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:21.946944 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:21.947613 23934 raft_consensus.cc:2376] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9. Re-sending same reply.
I20260709 12:14:21.927364 23926 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:14:21.926613 23930 raft_consensus.cc:1217] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Deduplicated request from leader. Original: 5.4->[6.5-8.6]   Dedup: 6.5->[8.6-8.6]
I20260709 12:14:21.943115 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:21.948380 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.948799 23925 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.949065 23931 raft_consensus.cc:2368] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6.
I20260709 12:14:21.949368 23930 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:14:21.924136 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.951686 27101 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.953905 27101 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:21.922885 27090 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:21.910458 27050 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:21.956583 25172 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:21.922710 27034 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.958923 27026 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:21.915536 27092 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:21.946954 27021 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:21.951480 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:21.956684 27050 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:21.966739 27021 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:21.967372 26718 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:21.967700 27021 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:21.969102 27021 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:21.914958 26788 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:21.974680 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 307962us
I20260709 12:14:21.977701 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:21.979089 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:21.978824 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 151507us
I20260709 12:14:21.987399 25171 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:22.004027 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:22.007316 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.032400 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.034162 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.034626 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:22.036532 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.037242 23857 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:22.039919 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.040665 23854 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:22.007393 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.048055 23852 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:22.035827 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:22.158470 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 122743us
I20260709 12:14:22.169562 23778 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:22.188038 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.985589 27022 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:22.189033 23784 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.191139 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.193813 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.194129 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.199252 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:22.200758 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.007544 27101 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:22.204960 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:22.205991 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.206530 23778 raft_consensus.cc:2376] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6. Re-sending same reply.
I20260709 12:14:22.208688 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.209156 23777 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:22.210146 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.211004 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.211531 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:22.212214 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.213106 23778 raft_consensus.cc:2376] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6. Re-sending same reply.
I20260709 12:14:22.214030 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.214793 23778 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.002218 27059 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:22.213205 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:21.989004 27026 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.217s
I20260709 12:14:22.032650 27092 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:22.032208 27090 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:22.222357 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.219259 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.244026 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:22.215332 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 349858us
I20260709 12:14:22.195362 27003 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.229986 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:22.247963 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.253131 26641 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.254021 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.254201 26788 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.255322 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.256153 23778 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.200656 27041 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:22.268453 27041 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.270721 27041 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.271853 27125 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:14:22.267762 27042 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:14:22.272413 26641 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.241650 27068 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.273342 27041 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:22.273744 27041 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:22.274561 26641 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:22.274502 27041 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.221688 26761 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:22.281450 27092 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:22.281931 26975 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.282680 27092 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.256963 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:22.257369 27076 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:22.264909 27004 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.267899 27111 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:22.284520 23783 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.284968 23778 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:22.289855 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.281980 23925 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:22.244692 23784 raft_consensus.cc:2376] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:14:22.291991 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.292565 23784 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.287513 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.293820 23931 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:22.290527 23778 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.305799 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:22.311648 23825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.012s	sys 0.033s
W20260709 12:14:22.312147 23825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.012s	sys 0.033s
W20260709 12:14:22.312471 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 581642us
I20260709 12:14:22.313531 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.286443 26761 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:22.289053 27103 consensus_meta.cc:287] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.001s
I20260709 12:14:22.317533 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.339437 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.339437 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.343570 23783 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.353106 23931 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:22.303983 27003 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.360052 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 502404us
W20260709 12:14:22.331974 27099 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.000s
I20260709 12:14:22.361663 23778 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
W20260709 12:14:22.362479 23899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.094s (SENT) [suppressed 107 similar messages]
W20260709 12:14:22.366288 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52835us
W20260709 12:14:22.364468 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 47023.4us
I20260709 12:14:22.374759 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.331207 27125 consensus_queue.cc:1243] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:14:22.380378 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:22.357460 26981 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.002s
I20260709 12:14:22.340444 27120 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:22.392442 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.402252 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.364768 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.403645 23782 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:22.401315 23783 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:22.403441 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.405520 23778 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.407051 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.408082 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:22.411405 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 146920us
I20260709 12:14:22.364333 27058 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:22.415373 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 217645us
I20260709 12:14:22.416800 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:22.411753 23933 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.029s	sys 0.015s
I20260709 12:14:22.290576 27059 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.305637 27004 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.403303 27125 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:22.400393 27120 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:22.424381 23933 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.030s	sys 0.015s
I20260709 12:14:22.340773 27103 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.426613 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.404263 27050 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:22.461742 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52442.6us
I20260709 12:14:22.425768 25114 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:22.483062 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.485877 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.505878 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.411520 26975 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.371742 27003 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:14:22.396960 27122 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:14:22.513057 23777 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 4.
I20260709 12:14:22.410403 27100 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:22.418335 27090 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:22.503177 23782 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:22.404371 27104 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 12:14:22.510789 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 85098.3us
I20260709 12:14:22.515666 26641 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:22.525334 25114 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:22.403184 26981 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.517582 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.424170 27101 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:22.528206 23782 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.533417 23857 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:22.414945 26761 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.536787 27101 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:22.538338 27021 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.000s
I20260709 12:14:22.539206 23779 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:14:22.539562 23779 pending_rounds.cc:85] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 4
I20260709 12:14:22.542168 23779 pending_rounds.cc:107] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:14:22.542450 23779 raft_consensus.cc:2889] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.542764 23779 pending_rounds.cc:107] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:14:22.542992 23779 raft_consensus.cc:2889] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.482306 26970 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:14:22.506923 27120 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.505220 27119 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:22.511567 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.547775 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.565606 23781 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
I20260709 12:14:22.565878 23780 raft_consensus.cc:2368] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
I20260709 12:14:22.533900 26641 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:22.569584 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.531734 27058 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:22.567291 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:22.566963 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:22.574434 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 211964us
I20260709 12:14:22.567642 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:22.568840 27114 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.694s	user 0.007s	sys 0.008s
W20260709 12:14:22.576128 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 416329us
I20260709 12:14:22.583338 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.576272 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.588567 23784 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:22.544700 26899 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:22.570266 26718 consensus_meta.cc:287] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.000s
I20260709 12:14:22.593045 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:22.600811 23931 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:22.600894 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:22.570863 26641 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:22.582134 27104 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:22.583518 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:22.544286 27090 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.596419 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.637830 23925 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:22.597088 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:22.598349 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.590111 23780 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:22.604818 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:22.605732 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:22.640556 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:22.624907 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:22.540568 26981 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.639859 23932 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:22.635795 26718 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:22.589105 27058 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:22.644907 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.645009 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:22.638505 23934 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:22.600278 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:22.646070 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:22.646064 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 126880us
I20260709 12:14:22.635319 23781 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:22.647198 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:22.628343 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 220313us
I20260709 12:14:22.648953 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:22.607411 27021 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:22.651866 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.624408 27050 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:22.660269 23858 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:14:22.660001 23934 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:22.657426 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.606043 27104 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:22.606137 26970 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:22.659253 23925 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:22.624466 26899 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:22.668843 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.644644 26718 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.663272 27050 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.671926 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:22.643752 27126 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:22.685513 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:22.711011 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:22.691744 25174 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.693920 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:22.694785 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:22.696278 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.696352 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:22.698483 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 332) took 1009 ms. Trace:
I20260709 12:14:22.714422 23933 rpcz_store.cc:276] 0709 12:14:21.688876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.688984 (+   108us) service_pool.cc:224] Handling call
0709 12:14:21.689372 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.689621 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:21.865896 (+176275us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.645863 (+779967us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.698101 (+ 52238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.698156 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:22.698210 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.698283 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.698414 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63451,"raft.queue_time_us":73939,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":21264,"thread_start_us":744097,"threads_started":3}
I20260709 12:14:22.659965 27021 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.633877 27136 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:22.711915 23853 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.685684 26981 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:22.711117 23925 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 4.
I20260709 12:14:22.719566 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.791522 23776 raft_consensus.cc:2376] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8. Re-sending same reply.
I20260709 12:14:22.681535 27099 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:22.670253 26970 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:22.717108 23860 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:22.790196 23856 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:14:22.790120 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 165324us
I20260709 12:14:22.653972 26831 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:14:22.800918 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.800918 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:22.721575 27131 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:14:22.807542 23858 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:22.715790 27059 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:22.819929 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.830662 23776 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:22.686293 27119 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:14:22.833108 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:22.789953 27021 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.850492 27016 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:22.855020 26718 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:22.856743 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.857374 27140 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 12:14:22.858134 27137 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:22.859427 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 58608.8us
W20260709 12:14:22.859820 27128 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:14:22.860288 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:22.861747 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:22.709162 27050 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.790788 26981 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.743929 27103 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.794701 26572 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.829356 23853 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:22.807121 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 480) took 1579 ms. Trace:
I20260709 12:14:22.834069 27100 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.830224 27130 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
W20260709 12:14:22.839919 27127 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 12:14:22.812036 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:22.807871 27058 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.802273 27004 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.808053 27104 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.845573 26970 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.846587 27059 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:22.820907 27099 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:22.845564 23776 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:22.848651 26899 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.836781 23858 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:22.864609 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 369) took 1907 ms. Trace:
W20260709 12:14:22.865950 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 164609us
I20260709 12:14:22.933930 23859 rpcz_store.cc:276] 0709 12:14:20.957603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.957732 (+   129us) service_pool.cc:224] Handling call
0709 12:14:22.864382 (+1906650us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:22.864540 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620322048}
I20260709 12:14:22.933872 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.713780 26761 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:22.856638 27090 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:22.833873 27026 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:22.864172 27054 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:22.868417 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 183174us
I20260709 12:14:22.935364 23776 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:22.894439 27068 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:22.937594 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 292643us
I20260709 12:14:22.720454 27126 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.891319 26981 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:22.937026 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.923820 27153 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.948742 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46282.1us
I20260709 12:14:22.960953 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:22.949832 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:22.951650 23930 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:22.962015 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:22.966367 26970 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.952217 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:22.952431 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:22.968797 25175 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:22.969785 27119 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:22.952829 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:22.957044 27139 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:14:22.957150 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.993618 23776 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:22.994633 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.995244 23776 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:22.954422 27103 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:22.983871 27132 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 12:14:22.903534 26572 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.997241 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2680) took 1262 ms. Trace:
I20260709 12:14:22.997645 23851 rpcz_store.cc:276] 0709 12:14:21.734481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.734642 (+   161us) service_pool.cc:224] Handling call
0709 12:14:21.735013 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.917862 (+182849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.917887 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:21.943715 (+ 25828us) log.cc:844] Serialized 26 byte log entry
0709 12:14:22.332813 (+389098us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:22.332856 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.332930 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.997002 (+664072us) raft_consensus.cc:1685] finished
0709 12:14:22.997023 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.997174 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":219091,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":529607,"threads_started":2,"wal-append.queue_time_us":545909}
W20260709 12:14:23.000488 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54405.6us
I20260709 12:14:23.002851 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.005393 27158 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:23.006842 27158 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:22.944670 27054 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:23.007002 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:23.007925 27054 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.009186 27054 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.009740 27121 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:23.014789 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:22.983850 27026 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.954288 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:22.972906 27135 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:23.021910 27135 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:23.009280 27041 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.022830 27135 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.009351 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:23.026384 27164 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:22.983996 27149 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.026973 27164 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.027379 27164 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:23.027364 27149 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.985098 26641 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:22.986364 27147 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:14:22.986006 27153 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:22.988370 27126 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:22.991285 23778 raft_consensus.cc:2410] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:22.991277 27099 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:22.991945 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:23.014889 27142 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:23.017777 23931 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:22.976841 27117 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.988s	user 0.005s	sys 0.006s
I20260709 12:14:23.042160 26641 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:23.018859 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66111.8us
I20260709 12:14:23.046252 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 327) took 1366 ms. Trace:
I20260709 12:14:23.046708 23928 rpcz_store.cc:276] 0709 12:14:21.680073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.680213 (+   140us) service_pool.cc:224] Handling call
0709 12:14:21.709969 (+ 29756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.710176 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.710200 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:21.710680 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:14:21.897205 (+186525us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:21.897249 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.897323 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.976107 (+1078784us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:23.045741 (+ 69634us) raft_consensus.cc:1685] finished
0709 12:14:23.045781 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.046149 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542336,"thread_start_us":186126,"threads_started":1,"wal-append.queue_time_us":277146}
I20260709 12:14:23.019395 26979 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:23.048588 26979 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:23.049647 26979 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:23.020203 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.020921 27103 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.053733 27103 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:23.033491 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72652.3us
W20260709 12:14:23.034237 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 72329.1us
I20260709 12:14:23.060817 27149 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.060861 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:23.061784 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:23.062932 23857 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:23.063328 23857 raft_consensus.cc:614] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:23.064402 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:23.036134 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.032636 23783 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
I20260709 12:14:23.066248 23776 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:23.065778 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:23.072876 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.073488 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:23.075249 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:22.905282 27100 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.074249 23776 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.076512 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:22.900821 25170 rpcz_store.cc:276] 0709 12:14:21.235018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.236988 (+  1970us) service_pool.cc:224] Handling call
0709 12:14:21.318977 (+ 81989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.588588 (+269611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.719458 (+1130870us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001e8054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:22.800237 (+ 80779us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492364672}
I20260709 12:14:23.082055 23928 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:14:23.085662 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49618.1us
W20260709 12:14:23.090683 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.032502 27080 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.091611 27080 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.092499 23776 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:23.093618 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:23.094357 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:23.080684 27122 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.085108 27121 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.073163 26748 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:23.096138 26967 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.074887 27068 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.097066 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.099721 27153 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.101899 27100 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.103757 26826 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:23.104919 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.103777 26718 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:23.112810 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:23.164052 23778 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:14:23.164250 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:23.164865 23783 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
W20260709 12:14:23.164988 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67999.7us
I20260709 12:14:23.165740 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.166288 23783 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:23.167117 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:23.167816 23782 raft_consensus.cc:2376] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5. Re-sending same reply.
I20260709 12:14:23.116779 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:23.170234 23784 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.171804 26826 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:23.118006 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2420) took 1091 ms. Trace:
I20260709 12:14:23.173480 23777 rpcz_store.cc:276] 0709 12:14:22.026960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.211971 (+185011us) service_pool.cc:224] Handling call
0709 12:14:23.117787 (+905816us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540012cad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:23.117932 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690230528}
I20260709 12:14:23.126304 25173 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:23.129006 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:23.063910 27059 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:23.084501 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2415) took 1061 ms. Trace:
I20260709 12:14:23.178115 23779 rpcz_store.cc:276] 0709 12:14:22.023397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.025505 (+  2108us) service_pool.cc:224] Handling call
0709 12:14:22.193294 (+167789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.543442 (+350148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.084223 (+540781us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54001eebd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:23.084427 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794706304}
I20260709 12:14:23.179445 26869 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:23.186467 23784 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:23.084178 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:23.042307 27142 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:23.063639 27090 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:23.073892 27103 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:23.147744 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 53479.1us
I20260709 12:14:23.074842 25168 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:23.067365 27119 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.155550 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:23.096944 26955 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.158560 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:23.114190 26981 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.167351 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.205870 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:23.217674 23931 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
W20260709 12:14:23.233143 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 149050us
I20260709 12:14:23.234948 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:23.145521 27016 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.167769 27126 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:23.237051 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:23.238960 23776 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:23.244051 23781 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:23.236169 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 165) took 6138 ms. Trace:
I20260709 12:14:23.158406 27101 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:23.131394 27004 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:14:23.081284 27120 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.155396 27122 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.158102 27121 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.194923 26967 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.288664 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:23.664415 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:23.262360 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 138010us
W20260709 12:14:23.484866 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.615681 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 449) took 1334 ms. Trace:
I20260709 12:14:23.473212 25114 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:23.668694 23749 leader_election.cc:400] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6.
I20260709 12:14:23.457386 26993 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.667711 23780 rpcz_store.cc:276] 0709 12:14:22.280923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.639413 (+358490us) service_pool.cc:224] Handling call
0709 12:14:23.118409 (+478996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.118607 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.118636 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:23.118676 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:23.118704 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.118766 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.615427 (+496661us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b540012cad0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:23.615605 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051385600}
I20260709 12:14:23.669778 23749 leader_election.cc:403] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:23.455016 25173 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:23.671103 27121 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:23.671840 25173 raft_consensus.cc:606] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:23.682268 23782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.009s	sys 0.013s
W20260709 12:14:23.684770 23782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.009s	sys 0.013s
I20260709 12:14:23.458524 23852 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:23.662822 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:23.697547 23852 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:23.259770 23776 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:23.699528 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2410) took 1047 ms. Trace:
I20260709 12:14:23.699963 23776 rpcz_store.cc:276] 0709 12:14:22.652570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.259110 (+606540us) service_pool.cc:224] Handling call
0709 12:14:23.259434 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.699196 (+439762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.699455 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:23.404567 23853 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:23.330363 23855 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:23.284211 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116877us
I20260709 12:14:23.284808 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:23.511826 27142 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.698482 23852 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.706228 23852 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:23.678129 27058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.025s
I20260709 12:14:23.259519 23680 rpcz_store.cc:276] 0709 12:14:17.069948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.070489 (+   541us) service_pool.cc:224] Handling call
0709 12:14:23.208143 (+6137654us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:18.716578 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:18.885515 (+168937us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:18.885581 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:14:18.885607 (+    26us) tablet.cc:662] Decoding operations
0709 12:14:18.886853 (+  1246us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:18.886937 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:18.886967 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:14:19.009814 (+122847us) tablet.cc:701] Row locks acquired
0709 12:14:19.036748 (+ 26934us) write_op.cc:260] PREPARE: finished
0709 12:14:19.064191 (+ 27443us) write_op.cc:270] Start()
0709 12:14:19.064371 (+   180us) write_op.cc:276] Timestamp: P: 1783599259064122 usec, L: 0
0709 12:14:19.064395 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:14:19.065452 (+  1057us) log.cc:844] Serialized 4010 byte log entry
0709 12:14:20.434010 (+1368558us) op_driver.cc:464] REPLICATION: finished
0709 12:14:20.722665 (+288655us) write_op.cc:301] APPLY: starting
0709 12:14:20.736785 (+ 14120us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:20.806220 (+ 69435us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:20.817686 (+ 11466us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:20.967389 (+149703us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:21.021950 (+ 54561us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:14:21.021999 (+    49us) write_op.cc:312] APPLY: finished
0709 12:14:21.022377 (+   378us) log.cc:844] Serialized 77 byte log entry
0709 12:14:21.022579 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:21.022713 (+   134us) write_op.cc:454] Released schema lock
0709 12:14:21.022778 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:21.023239 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:21.023266 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":582407,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":242194,"apply.run_cpu_time_us":5878,"apply.run_wall_time_us":322644,"num_ops":10,"prepare.queue_time_us":491662,"prepare.run_cpu_time_us":51614,"prepare.run_wall_time_us":768219,"raft.queue_time_us":237993,"raft.run_cpu_time_us":10582,"raft.run_wall_time_us":147716,"replication_time_us":1364338,"thread_start_us":749954,"threads_started":4,"wal-append.queue_time_us":481273,"wal-append.run_cpu_time_us":19646,"wal-append.run_wall_time_us":2474647}]]}
W20260709 12:14:23.687644 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 399026us
I20260709 12:14:23.313640 26869 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.695338 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:23.466043 27080 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:23.654860 26970 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.001s
W20260709 12:14:23.659806 27050 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.764s	user 0.004s	sys 0.008s
I20260709 12:14:23.704097 23859 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:23.704319 23776 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:23.663269 23781 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:23.795730 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2412) took 1142 ms. Trace:
I20260709 12:14:23.710889 23860 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:23.489581 27065 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:23.701752 26748 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.800305 27065 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:23.802276 26967 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.710965 27099 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:23.804245 27004 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:23.804119 27099 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:23.804791 27004 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:23.806066 27099 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.444017 27165 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.237679 27100 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:23.670326 26993 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:23.313946 26831 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:23.662640 27104 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.518549 26981 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:23.418493 27156 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:23.464426 27160 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:23.329020 26761 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.642373 26572 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.484927 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 741) took 1508 ms. Trace:
I20260709 12:14:23.331259 27145 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.403569 27021 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.302932 27092 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:23.796093 23781 rpcz_store.cc:276] 0709 12:14:22.653023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.291938 (+638915us) service_pool.cc:224] Handling call
0709 12:14:23.663088 (+371150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.795490 (+132402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.795671 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:23.799436 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:23.799804 26592 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:23.801638 27065 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.794534 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:23.704295 23900 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:23.782122 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 308990us
I20260709 12:14:23.724336 23779 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:14:23.878391 23779 pending_rounds.cc:85] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 2
I20260709 12:14:23.878679 23779 pending_rounds.cc:107] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:14:23.878933 23779 raft_consensus.cc:2889] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.878415 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:23.765028 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.811836 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.809545 23852 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:14:23.882083 23852 pending_rounds.cc:85] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 6
I20260709 12:14:23.882349 23852 pending_rounds.cc:107] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:14:23.882572 23852 raft_consensus.cc:2889] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.883291 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2687) took 1465 ms. Trace:
I20260709 12:14:23.799273 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:23.819193 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 123953us
I20260709 12:14:23.884697 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:23.245859 27090 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:23.723320 27058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.009s	sys 0.025s
I20260709 12:14:23.708118 27054 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:23.913867 27054 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.916841 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:23.918332 27021 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:23.918802 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2411) took 1266 ms. Trace:
I20260709 12:14:23.825078 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 377) took 1143 ms. Trace:
W20260709 12:14:23.919904 27111 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 12:14:23.828739 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.280797 27016 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.833245 26995 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:23.925082 26995 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.925992 27016 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.926455 27080 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:23.919225 23785 rpcz_store.cc:276] 0709 12:14:22.652798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.014512 (+361714us) service_pool.cc:224] Handling call
0709 12:14:23.918575 (+904063us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54003705d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:23.918735 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072248704}
I20260709 12:14:23.819272 27100 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:23.928282 23778 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:23.928854 27058 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.723668 26718 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.714049 27059 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6.
I20260709 12:14:23.708385 27101 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:23.930781 26993 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:23.257782 27119 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.845039 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:23.931977 23930 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.723467 27133 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.931313 27101 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.934118 27133 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.723367 26975 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.935931 26975 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.837378 27162 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 12:14:23.937049 27054 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.836666 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:23.900681 23933 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:14:23.942011 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 232925us
I20260709 12:14:23.942891 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.944096 27065 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:23.945057 27120 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.815220 27142 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:23.898236 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:23.806192 27004 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.873783 27145 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.837728 26641 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:23.911223 27090 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.821758 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:23.842679 25172 rpcz_store.cc:276] 0709 12:14:21.976107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.978636 (+  2529us) service_pool.cc:224] Handling call
0709 12:14:22.217029 (+238393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.271850 (+ 54821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.271884 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.271937 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:22.271972 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.484504 (+1212532us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54006d0e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:23.484582 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.484811 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113333504}
I20260709 12:14:23.912942 26731 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:23.831489 27159 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:23.929008 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 44404.6us
I20260709 12:14:23.932559 27122 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.976912 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:23.946501 27068 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:23.945850 27124 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:23.949517 27158 consensus_meta.cc:287] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.942s	user 0.008s	sys 0.006s
W20260709 12:14:23.886379 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.919750 23855 rpcz_store.cc:276] 0709 12:14:22.681664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.681874 (+   210us) service_pool.cc:224] Handling call
0709 12:14:22.687964 (+  6090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.703352 (+1015388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.824844 (+121492us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54002df354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:23.825015 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476848896}
I20260709 12:14:23.952651 23930 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:23.884995 23683 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:23.994108 23683 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:23.885428 23852 rpcz_store.cc:276] 0709 12:14:22.418193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.420166 (+  1973us) service_pool.cc:224] Handling call
0709 12:14:22.420414 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.458400 (+1037986us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000904a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad729dc0 00007fb2ad7b41c0 00007fb2ad7b5231 00007fb2ad7b1d8f 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c
0709 12:14:23.883043 (+424643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.883229 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234170752}
W20260709 12:14:23.889897 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 185684us
W20260709 12:14:23.951731 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73549.7us
I20260709 12:14:23.966346 26641 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.970036 26731 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.992436 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643746a7a11243df94746cf1081fc078"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:23.998446 26731 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:23.954773 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.814749 26748 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.025540 26641 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.026491 26748 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.962859 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:23.848857 27156 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:24.028344 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65562.4us
I20260709 12:14:24.029109 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.964442 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.030325 27145 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.030582 27156 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:24.030359 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.848670 27160 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.928504 27100 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:23.939013 26975 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.974278 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:23.975081 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:23.966717 23851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.019s	sys 0.031s
I20260709 12:14:23.977672 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.945849 27119 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.836097 26831 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:23.997615 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:23.811656 27050 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:23.971673 27175 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.092s	user 0.001s	sys 0.004s
I20260709 12:14:24.021165 26981 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:14:24.019055 23857 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:23.964097 26899 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:23.879650 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2414) took 1226 ms. Trace:
I20260709 12:14:23.822224 27165 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:23.958063 27136 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.011119 23683 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.114377 23751 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:24.024575 27003 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:23.872920 26761 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.103852 26748 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:23.929733 26718 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.105661 26869 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:24.031757 27103 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:24.073244 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52336.9us
I20260709 12:14:24.076604 27068 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:23.950928 27092 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.031414 27156 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:24.114826 26899 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.083678 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:24.085641 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.019330 25170 raft_consensus.cc:606] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:24.076144 23852 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:24.026355 25168 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:24.093127 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:23.901695 26592 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:24.138062 27164 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.110s	user 0.010s	sys 0.005s
W20260709 12:14:24.209229 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.101346 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 123749us
I20260709 12:14:24.002354 27120 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.078052 27134 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:24.086856 27041 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.064s	user 0.011s	sys 0.004s
I20260709 12:14:24.113890 23779 rpcz_store.cc:276] 0709 12:14:22.653480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.695290 (+1041810us) service_pool.cc:224] Handling call
0709 12:14:23.723947 (+ 28657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.879395 (+155448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.879574 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:24.117352 27080 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:14:24.141453 26899 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:24.072940 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.235904 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 157589us
I20260709 12:14:24.126994 27021 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:24.132839 27160 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.245273 23858 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:24.133649 26748 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:24.133512 26731 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.134742 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.249404 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.083415 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:23.993848 27159 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.147046 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:24.171319 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.097151 23851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.019s	sys 0.031s
I20260709 12:14:24.258123 23857 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:24.207491 23860 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:24.113571 27090 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:14:24.100235 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.114765 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2448) took 1805 ms. Trace:
I20260709 12:14:24.137123 27103 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:24.003762 27158 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:24.216178 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 70603.5us
I20260709 12:14:24.267498 27103 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.121065 23683 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.268760 27103 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:24.269407 23683 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 7, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.238166 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:24.272177 23899 leader_election.cc:400] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
W20260709 12:14:24.210760 27179 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:14:24.262447 23782 rpcz_store.cc:276] 0709 12:14:22.309395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.168510 (+859115us) service_pool.cc:224] Handling call
0709 12:14:23.168912 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.169154 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:23.169568 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.705827 (+536259us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.942852 (+237025us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:24.105278 (+162426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.105340 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.105391 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.105461 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.114690 (+  9229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56223,"prepare.queue_time_us":9216,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":83,"raft.queue_time_us":242323,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":12626,"spinlock_wait_cycles":27361024,"thread_start_us":913215,"threads_started":2,"wal-append.queue_time_us":666090}
I20260709 12:14:24.206517 26730 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:24.240806 27026 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.225s	user 0.009s	sys 0.007s
I20260709 12:14:24.129756 27136 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.275034 27133 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:24.278594 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68167.4us
I20260709 12:14:24.239616 27021 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:24.279277 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 205179us
I20260709 12:14:24.271193 23683 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 6, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 6 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:24.281064 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.257534 27119 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.289304 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.289006 27090 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.291517 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.278791 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:24.262109 27185 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.213743 27164 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.300024 26592 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.286947 25176 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:24.281736 25169 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:24.304162 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:24.329620 27152 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.330351 27152 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.331693 26981 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:24.332552 26981 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:24.334033 26981 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.335930 26761 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.336856 27176 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.337752 27176 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.297084 27026 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:24.296321 27041 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:14:24.344317 27041 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:24.344883 27041 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:24.346096 27041 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.347752 27176 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.354135 27164 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:24.358583 26993 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.359383 26993 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:24.362259 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.362874 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:24.304418 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.237098 27120 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.298439 23683 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 4, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 4 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:24.365356 23683 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:24.304057 23853 raft_consensus.cc:2410] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:14:24.308162 26831 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.308557 23782 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:24.308813 26748 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.309429 27090 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.313233 26572 raft_consensus.cc:993] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:24.314286 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 460) took 1201 ms. Trace:
I20260709 12:14:24.383751 23780 rpcz_store.cc:276] 0709 12:14:23.112788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.706373 (+593585us) service_pool.cc:224] Handling call
0709 12:14:23.706634 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.211083 (+504449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.211119 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.211162 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.211189 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.314008 (+102819us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54003714d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:24.314056 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.314217 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159681920}
I20260709 12:14:24.293484 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:24.282212 26869 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:24.316553 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.307058 27092 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:24.369508 27180 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
I20260709 12:14:24.371105 23683 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
W20260709 12:14:24.385646 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 92226.2us
I20260709 12:14:24.297598 27021 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:24.289472 27156 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:14:24.393484 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2447) took 2084 ms. Trace:
I20260709 12:14:24.397626 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:24.367816 27041 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.403820 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:24.400947 23783 rpcz_store.cc:276] 0709 12:14:22.309173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.166957 (+857784us) service_pool.cc:224] Handling call
0709 12:14:23.167271 (+   314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:23.257236 (+ 89965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.257268 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:23.696832 (+439564us) log.cc:844] Serialized 52 byte log entry
0709 12:14:24.211939 (+515107us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.237770 (+ 25831us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.292156 (+ 54386us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.292378 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.292449 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.393227 (+100778us) raft_consensus.cc:1685] finished
0709 12:14:24.393249 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.393412 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116146,"prepare.queue_time_us":43561,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":58,"raft.queue_time_us":131740,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":45726,"spinlock_wait_cycles":18146816,"thread_start_us":406769,"threads_started":1,"wal-append.queue_time_us":502081}
I20260709 12:14:24.380939 27167 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.412351 23857 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:24.382905 27164 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.383104 26761 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:24.383291 27026 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.318568 27119 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:24.426249 25176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 256) took 1868 ms (client timeout 1516 ms). Trace:
I20260709 12:14:24.411312 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:24.390763 27016 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:24.405006 23683 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 5 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:24.498606 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 87384.7us
W20260709 12:14:24.502876 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.499425 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 7, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:24.510617 23683 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.512348 23683 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 4 to 5, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 5 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.514123 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 8, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:24.515743 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:24.516455 23852 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:24.363108 27185 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:24.498934 25176 rpcz_store.cc:269] 0709 12:14:22.558252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.558363 (+   111us) service_pool.cc:224] Handling call
0709 12:14:24.426028 (+1867665us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54001e87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:24.426164 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3831630720}
I20260709 12:14:24.498323 27104 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.519555 27104 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.520586 27185 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.402634 27176 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.523980 27176 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.525277 25176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:24.525946 25176 raft_consensus.cc:606] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:24.412240 27142 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:24.422719 27178 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 12:14:24.424201 27183 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 12:14:24.528136 27185 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.416990 27120 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.529027 27185 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.431574 27167 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.532570 27185 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.533039 27167 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:24.421522 27181 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:14:24.535812 27182 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:24.536584 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 388) took 1303 ms. Trace:
I20260709 12:14:24.536964 23854 rpcz_store.cc:276] 0709 12:14:23.233178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.233557 (+   379us) service_pool.cc:224] Handling call
0709 12:14:23.233899 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.672571 (+438672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.672590 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.672954 (+   364us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.975278 (+302324us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:23.975314 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.975384 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.420888 (+445504us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:24.536199 (+115311us) raft_consensus.cc:1685] finished
0709 12:14:24.536223 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.536520 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46033,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":25723,"spinlock_wait_cycles":264866688,"thread_start_us":739893,"threads_started":2,"wal-append.queue_time_us":427128}
I20260709 12:14:24.538945 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:24.478464 27164 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.543071 27164 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.536617 27182 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:24.545149 27182 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.412084 27160 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.136s
I20260709 12:14:24.547721 27119 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.539528 23854 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.549736 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 387) took 1318 ms. Trace:
I20260709 12:14:24.550225 26731 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:24.550125 23851 rpcz_store.cc:276] 0709 12:14:23.231377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.231945 (+   568us) service_pool.cc:224] Handling call
0709 12:14:23.232331 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.232922 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:23.232953 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.257276 (+ 24323us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.259228 (+1001952us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:24.259267 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.259330 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.422121 (+162791us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:24.549388 (+127267us) raft_consensus.cc:1685] finished
0709 12:14:24.549409 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.549668 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166,"raft.queue_time_us":259,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":44443,"spinlock_wait_cycles":99242752,"thread_start_us":857329,"threads_started":1,"wal-append.queue_time_us":821311}
I20260709 12:14:24.552368 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.559854 26641 raft_consensus.cc:1081] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:24.566365 23854 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
W20260709 12:14:24.572729 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.574187 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.575919 23928 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:24.577832 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.579288 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.580171 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.580614 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.584975 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.579995 23927 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:24.589219 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.648065 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.648500 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.589952 23928 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:24.552552 27119 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.669893 23854 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:24.528812 27090 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.715418 23854 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:14:24.715852 23854 pending_rounds.cc:85] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 3
I20260709 12:14:24.716087 23854 pending_rounds.cc:107] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:24.716316 23854 raft_consensus.cc:2889] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.587303 26731 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:24.745606 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.587123 27120 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:24.766842 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.768288 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 489533us
I20260709 12:14:24.509670 27026 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.771721 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:24.779615 23852 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:24.699911 27185 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.711722 27016 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:24.799247 23777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.136s	sys 0.229s
I20260709 12:14:24.713275 27134 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:24.811896 23777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.136s	sys 0.230s
I20260709 12:14:24.714550 27165 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:24.768134 26899 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 12:14:24.820470 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2449) took 2510 ms. Trace:
I20260709 12:14:24.408866 27092 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:24.827062 25173 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:24.825525 23777 rpcz_store.cc:276] 0709 12:14:22.309749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.174442 (+864693us) service_pool.cc:224] Handling call
0709 12:14:23.174786 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:23.292805 (+118019us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:23.377437 (+ 84632us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.454776 (+ 77339us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b18003adce0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:24.029246 (+574470us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b18003adce0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:24.118190 (+ 88944us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:24.118869 (+   679us) log.cc:844] Serialized 52 byte log entry
0709 12:14:24.119053 (+   184us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:24.119271 (+   218us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.819827 (+700556us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.820090 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.820175 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.820228 (+    53us) raft_consensus.cc:1685] finished
0709 12:14:24.820241 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.820396 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":468161,"prepare.queue_time_us":109128,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":2046,"raft.queue_time_us":372237,"raft.run_cpu_time_us":620,"raft.run_wall_time_us":93057,"spinlock_wait_cycles":176900736,"thread_start_us":1055791,"threads_started":3,"wal-append.queue_time_us":405878}
I20260709 12:14:24.838466 25176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:24.839092 25176 raft_consensus.cc:606] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:24.839483 27065 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.895s	user 0.014s	sys 0.000s
I20260709 12:14:24.836122 26899 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.837393 27092 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:24.848831 27092 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:24.849948 23930 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:24.855785 27159 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.858712 27026 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:24.859812 23931 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:24.793829 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 747) took 1586 ms. Trace:
I20260709 12:14:24.714249 27119 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.817049 27142 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.865558 23859 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:14:24.817939 27167 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:24.766376 26731 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.793020 27156 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:14:24.826576 27090 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:24.770448 27120 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.877558 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.830374 27016 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.825963 27185 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:24.886332 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.870872 27142 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:24.899560 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 127961us
I20260709 12:14:24.898074 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.904083 27065 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:24.904659 27065 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:24.905054 27065 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:24.908224 27134 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:24.908743 26731 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:24.911506 26899 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.873889 27167 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:24.925221 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.926201 23853 raft_consensus.cc:2410] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:24.926344 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.927479 23925 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:24.882750 27090 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:24.933018 23933 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:24.917140 27021 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.002s
W20260709 12:14:24.936539 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.008s	sys 0.016s
W20260709 12:14:24.927388 27196 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 12:14:24.936702 23925 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:24.938333 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.008s	sys 0.016s
W20260709 12:14:24.929291 26572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.012s	sys 0.011s
W20260709 12:14:24.952409 26572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.013s	sys 0.011s
I20260709 12:14:24.952574 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.965662 27165 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.973748 27165 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:24.939007 23825 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.012s	sys 0.012s
W20260709 12:14:24.976118 23825 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.012s	sys 0.012s
W20260709 12:14:24.976423 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 547525us
W20260709 12:14:24.970625 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.003s	sys 0.023s
W20260709 12:14:24.978081 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.003s	sys 0.024s
I20260709 12:14:24.980355 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:24.981000 23777 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:24.977380 23825 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.816s (SENT)
I20260709 12:14:24.988649 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:24.991118 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:24.991859 23783 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:24.993134 23785 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:24.993568 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:24.994199 23784 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:24.996519 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.997249 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:24.997884 23785 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.997972 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.000872 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.001492 23934 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:25.002202 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:24.971349 27120 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:24.968909 27103 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.010597 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:24.984133 27100 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.877s	user 0.004s	sys 0.009s
I20260709 12:14:25.008437 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.014982 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:25.015396 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.009090 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:25.009919 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.016242 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 91565.4us
I20260709 12:14:25.011649 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.012454 23784 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:25.005868 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:25.017843 23778 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.018146 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.018960 23785 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:25.014276 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.019297 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.021063 23934 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:25.021974 23934 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:14:25.013300 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:25.024827 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.024415 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:25.023720 23783 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:25.032974 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:25.017144 27119 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:24.939198 27021 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.042109 23778 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:25.044196 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.044489 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.045030 25175 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:25.045715 25175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:25.047114 25175 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:25.048071 23855 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:25.006841 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:25.048799 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.049182 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.051275 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.052181 25176 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:14:25.054188 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.055053 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61190.9us
I20260709 12:14:25.056602 27119 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:25.056891 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.060053 25172 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:25.061899 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.048599 26731 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:25.064101 26731 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.064956 27090 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.043182 27159 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:14:25.068102 27159 consensus_queue.cc:1243] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:24.863045 25171 rpcz_store.cc:276] 0709 12:14:23.182869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.183053 (+   184us) service_pool.cc:224] Handling call
0709 12:14:23.183530 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.193674 (+ 10144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.230228 (+ 36554us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.519887 (+289659us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.994856 (+474969us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:23.994907 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.994994 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.368578 (+373584us) spinlock_profiling.cc:243] Waited 50.9 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:24.670969 (+302391us) raft_consensus.cc:1685] finished
0709 12:14:24.700079 (+ 29110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.769645 (+ 69566us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14762,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5963,"spinlock_wait_cycles":117005056,"thread_start_us":468721,"threads_started":1,"wal-append.queue_time_us":599812}
I20260709 12:14:25.068722 27159 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:25.042905 27204 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.069558 27204 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.068259 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.070371 27050 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.070580 26993 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.065901 27034 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:25.074910 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.075192 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.026942 27201 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:25.170549 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.127274 23778 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:25.039583 27021 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.070183 27065 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.135797 27016 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.136168 26993 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:25.077086 27101 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:14:25.069073 27159 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:25.069070 27104 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.043558 27165 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:25.057694 27195 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:14:25.183507 27204 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.192543 23928 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:25.027742 27103 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:25.196210 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 121095us
W20260709 12:14:25.197098 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 196231us
I20260709 12:14:25.028090 27120 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:14:25.225013 27142 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:25.222009 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.058336 27197 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.200023 26592 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.080157 27164 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.151s
I20260709 12:14:25.202759 26993 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.193836 27065 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.018533 27100 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.228361 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:25.251354 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 194513us
I20260709 12:14:25.252514 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.231025 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.244264 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1103) took 1159 ms. Trace:
I20260709 12:14:25.253428 27021 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:25.253943 27021 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.237387 27068 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:25.254276 27021 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.253298 23860 rpcz_store.cc:276] 0709 12:14:24.085274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.085390 (+   116us) service_pool.cc:224] Handling call
0709 12:14:25.244071 (+1158681us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b54002f62d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:25.244192 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153411456}
I20260709 12:14:25.238451 27165 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:14:25.247936 26993 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.254395 27068 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:25.258181 27165 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:25.258587 27165 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:25.222992 27016 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.259425 27165 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.253422 26995 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.254827 23859 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:25.253165 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.259469 27016 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.251139 27167 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.196372 27201 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.262766 27016 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.246253 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.263909 23925 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:25.264103 23779 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:25.264971 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.261394 27160 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:25.268931 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:25.219655 27092 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.267179 23925 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:25.273706 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.276362 23925 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:25.274273 23856 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:14:25.278365 23856 pending_rounds.cc:85] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 6
I20260709 12:14:25.278717 23856 pending_rounds.cc:107] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:14:25.278887 23856 raft_consensus.cc:2889] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:25.270174 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:25.280344 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49407.7us
I20260709 12:14:25.272246 27160 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:25.282763 27160 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:25.283129 27160 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:25.272559 27119 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:25.285523 27119 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.287758 27142 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:25.288255 27142 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.271968 27204 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:25.289106 27204 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.289559 27204 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.288573 27119 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.288609 27142 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.281149 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.295254 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:25.296273 23777 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:25.296736 23777 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:25.296993 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:25.280066 27041 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.867s	user 0.011s	sys 0.000s
I20260709 12:14:25.282215 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:25.269619 27201 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.289016 27016 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:14:25.307207 27050 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:25.307675 27050 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:25.308521 27050 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.308887 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.309509 26592 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:25.309729 27050 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.304162 27197 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:25.310684 27197 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.310782 27136 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.311072 27197 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:25.299252 27200 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:25.313755 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:25.315079 27100 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.315678 25119 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.316660 26592 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:25.316897 27103 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.294483 27153 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:25.310140 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.317570 27200 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:25.299705 27165 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.325039 25174 raft_consensus.cc:2410] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:25.266880 26993 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:25.325786 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 43639.3us
I20260709 12:14:25.329859 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:25.331657 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1102) took 1247 ms. Trace:
W20260709 12:14:25.330417 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.334682 23858 rpcz_store.cc:276] 0709 12:14:24.084398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.084520 (+   122us) service_pool.cc:224] Handling call
0709 12:14:24.084846 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.245803 (+160957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.331373 (+1085570us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400025154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:25.331573 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368417024}
I20260709 12:14:25.332356 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.335907 23860 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:25.333292 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.336751 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:25.337688 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.338222 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.338814 23854 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:25.275954 27133 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:14:25.340020 23777 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.336817 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.342360 23858 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:25.341709 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.335640 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:25.266880 27021 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.344805 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.348538 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.343544 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.345798 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:25.350886 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.351634 23857 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:25.333714 27201 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:25.356269 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.356833 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.357813 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.358373 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.360404 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.361114 25176 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:25.362089 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.364310 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.365742 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.369357 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.344337 27016 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.342146 27041 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.377043 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.378867 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.367025 27201 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.381234 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.381608 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:25.384037 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.379575 23932 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:25.387745 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.392894 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.395677 23932 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.370321 27182 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:25.367664 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.371898 27080 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:25.370369 27103 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:25.373332 26748 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:25.407521 26748 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.408084 26748 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.368571 27200 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 12:14:25.410897 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.412043 27216 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:25.412727 27216 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.413157 27216 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:25.401396 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:25.389729 27201 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.405468 23751 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.362875 27090 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:14:25.410334 26748 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.400420 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.377327 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.419997 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.428654 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.429175 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.402577 27050 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:25.402426 27182 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.436107 27182 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.436578 27136 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:25.439760 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.440982 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.441440 25177 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:25.441740 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.442015 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.442360 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.442845 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.442992 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.443925 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.445034 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.395288 26592 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:25.403494 27016 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.434756 27050 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:25.435360 27218 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:25.430189 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.415820 27100 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:25.443450 23926 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
W20260709 12:14:25.447077 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48862.4us
I20260709 12:14:25.445781 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:25.513266 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 83147.5us
I20260709 12:14:25.395181 27121 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:14:25.446234 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:25.466982 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.420540 25171 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:25.445616 27221 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:25.469165 27068 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:25.470820 25168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 491) took 1755 ms (client timeout 1913 ms). Trace:
I20260709 12:14:25.436110 27176 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.475658 25177 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:25.479750 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.553282 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.426820 27080 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:25.524416 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 78737.5us
I20260709 12:14:25.457194 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.558111 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.557684 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:25.571337 23925 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
W20260709 12:14:25.571466 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47469.1us
I20260709 12:14:25.570086 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.571163 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.574805 23928 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:25.575155 27103 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.475939 27160 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.572942 26955 raft_consensus.cc:993] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:25.523746 26981 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:25.574893 27090 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.594532 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.604003 23784 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:25.594730 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.595166 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.595446 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.569532 27119 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.462697 27142 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.625849 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.568835 27100 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:25.569758 27121 consensus_queue.cc:1243] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:14:25.589969 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.566846 26748 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:14:25.593889 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.554019 27204 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:25.596314 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:25.638955 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 165516us
I20260709 12:14:25.583158 23928 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:25.567088 27120 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:25.620009 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.641163 23782 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:25.642160 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:25.639520 23776 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:25.620733 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.643718 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.519876 27016 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.469255 26993 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.446900 27133 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:25.650725 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.435386 27092 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.637049 23785 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:25.590521 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 144389us
I20260709 12:14:25.645946 23783 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:25.643383 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:25.647375 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72587.8us
I20260709 12:14:25.611923 26981 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.669036 26981 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.670112 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.670346 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.649134 23776 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:25.671931 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.672431 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.673128 23776 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:25.591368 27103 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.601315 27228 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:25.659570 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:25.709357 23785 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.660885 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.710896 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:25.711151 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 50342.2us
I20260709 12:14:25.711874 23783 raft_consensus.cc:2410] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:14:25.667974 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.673960 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.647850 23928 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:25.641820 27021 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.651743 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.640883 27229 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:25.566156 25168 rpcz_store.cc:269] 0709 12:14:23.715837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.723447 (+  7610us) service_pool.cc:224] Handling call
0709 12:14:25.470628 (+1747181us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b540067e8d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:25.470753 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137053952}
I20260709 12:14:25.729928 27103 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.730407 27182 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.709494 27164 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:25.702823 23780 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:25.734750 27164 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:25.735054 27160 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.655222 27133 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.735952 27133 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:25.673353 27225 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.623095 26592 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.737166 27225 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.707827 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2700) took 1628 ms. Trace:
I20260709 12:14:25.739866 23855 rpcz_store.cc:276] 0709 12:14:24.079837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.079992 (+   155us) service_pool.cc:224] Handling call
0709 12:14:24.101471 (+ 21479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.048419 (+946948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.707497 (+659078us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b5400386b54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:25.707752 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176468736}
I20260709 12:14:25.663125 26899 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:25.712997 23928 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:25.636938 27041 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:25.655441 27120 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:25.763976 26981 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.764394 27120 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.733166 27142 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.632107 27050 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.717113 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.719210 26718 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.651472 27231 raft_consensus.cc:1081] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:25.721141 23785 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:25.768630 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 100047us
I20260709 12:14:25.734671 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 749) took 1431 ms. Trace:
W20260709 12:14:25.744248 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.633111 27119 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.745085 23782 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:25.708400 27210 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.527s	user 0.004s	sys 0.000s
I20260709 12:14:25.746377 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:25.742057 23928 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.786725 23928 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.712936 23778 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:25.728758 27016 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.739377 27225 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.779040 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.779430 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.796293 23783 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:25.797091 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:25.780584 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 128902us
W20260709 12:14:25.785353 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.799443 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:25.790999 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
W20260709 12:14:25.823493 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 77183.4us
I20260709 12:14:25.805063 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.828308 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.803238 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.833390 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.834976 23785 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:25.836028 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:25.838907 23776 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:14:25.839138 23928 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:14:25.840869 27068 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:25.843797 27068 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:25.777981 27229 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.845986 27068 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:25.792074 27160 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:25.847736 27160 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.848327 27160 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:25.849457 27034 consensus_meta.cc:287] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.004s
I20260709 12:14:25.841468 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.851742 23932 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 5: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:25.852150 23928 pending_rounds.cc:85] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 3
I20260709 12:14:25.852610 23928 pending_rounds.cc:107] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:14:25.778380 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:25.854485 23928 raft_consensus.cc:2889] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:25.855762 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:25.855988 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.780912 27230 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:25.858283 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58937us
I20260709 12:14:25.858886 27221 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:25.870368 23783 raft_consensus.cc:2368] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6.
I20260709 12:14:25.860800 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.920446 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:25.922263 23855 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:25.800400 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.925942 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.929355 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:25.930013 23853 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.870718 27230 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.869460 25172 raft_consensus.cc:2376] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5. Re-sending same reply.
I20260709 12:14:25.932349 27230 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:25.932833 27201 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 12:14:25.933480 27201 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.862839 27229 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.936563 23784 raft_consensus.cc:2368] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
W20260709 12:14:25.937369 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.921710 27221 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.939881 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.941196 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.942426 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:25.942927 23928 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:25.952471 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 150096us
I20260709 12:14:25.952929 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.954092 23857 raft_consensus.cc:2410] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:14:25.953954 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:25.962814 23853 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:25.943745 27197 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
I20260709 12:14:25.922272 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:25.928314 27231 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.831741 27133 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:25.969018 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:25.931275 27233 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:25.959723 27216 consensus_meta.cc:287] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 12:14:25.988106 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:25.988827 23855 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:25.936625 27182 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.992000 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:25.780607 25170 rpcz_store.cc:276] 0709 12:14:24.302781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.315402 (+ 12621us) service_pool.cc:224] Handling call
0709 12:14:25.475045 (+1159643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.508699 (+ 33654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.511540 (+  2841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:25.523347 (+ 11807us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:25.526468 (+  3121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.570859 (+ 44391us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.734476 (+163617us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540067e8d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:25.734591 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2646337280}
I20260709 12:14:26.012974 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.014142 23932 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:14:26.017479 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.019109 27201 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:25.997681 27216 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:25.984740 27065 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:14:25.980774 27134 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:14:26.026955 27133 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:25.993273 27233 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:25.858778 27100 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.004107 23928 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:26.020118 27248 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:25.992681 27238 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:14:26.057787 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.061478 27119 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.062207 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:26.064018 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:25.983071 27197 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.068212 23932 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:26.068717 23928 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:14:26.069587 23860 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:25.993283 27231 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.094215 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:26.051928 27233 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.099802 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.102916 27120 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:26.104884 27159 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.106568 27152 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.109042 25169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 260) took 3049 ms. Trace:
I20260709 12:14:26.109450 27197 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.110522 27230 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.111274 27242 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.117409 23825 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:26.122315 27104 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:26.129266 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.101317 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:26.117918 27120 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.147807 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:26.120379 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2421) took 1129 ms. Trace:
I20260709 12:14:26.156965 23781 rpcz_store.cc:276] 0709 12:14:24.991322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.992031 (+   709us) service_pool.cc:224] Handling call
0709 12:14:25.014930 (+ 22899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.015142 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:25.047393 (+ 32251us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.028632 (+981239us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b18003fcc40. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:26.119995 (+ 91363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.120055 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:26.120110 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.120183 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.120312 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":32102,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":13259,"raft.queue_time_us":110,"raft.run_cpu_time_us":2053,"raft.run_wall_time_us":10212,"spinlock_wait_cycles":2532608,"thread_start_us":91055,"threads_started":1}
I20260709 12:14:26.122653 27167 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.160034 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.134584 27159 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:26.162968 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.154749 27119 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:26.151360 27104 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:26.163395 27133 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.163895 26718 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:26.178171 26718 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.178560 26718 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.175189 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:26.180056 23781 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.159039 27120 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:26.179467 27136 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.180948 27136 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.183104 27136 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.183321 23928 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:26.153235 27242 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.185890 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:26.188767 23776 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:26.188237 27185 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.190250 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.190908 23928 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:26.192003 27239 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:26.179453 27104 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:26.198967 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:26.215430 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.216250 23925 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.171916 27119 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.234565 23925 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:26.217005 23781 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:26.218237 27216 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.188444 27242 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:26.220806 27197 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:26.227805 27103 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:26.243381 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:26.215633 27253 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 12:14:26.117650 25169 rpcz_store.cc:276] 0709 12:14:23.059436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.059630 (+   194us) service_pool.cc:224] Handling call
0709 12:14:23.060176 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.329393 (+1269217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.108783 (+1779390us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54004975d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:26.108977 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2800954112}
W20260709 12:14:26.245915 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1029) took 1672 ms (client timeout 1526 ms). Trace:
I20260709 12:14:26.251296 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.254451 23784 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 10. Ops: []
I20260709 12:14:26.249927 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:26.267179 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.275250 23781 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:14:26.275923 23781 pending_rounds.cc:85] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 6
I20260709 12:14:26.276197 23781 pending_rounds.cc:107] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:14:26.276522 23781 raft_consensus.cc:2889] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:26.243839 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.273331 23931 rpcz_store.cc:269] 0709 12:14:24.573761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.573905 (+   144us) service_pool.cc:224] Handling call
0709 12:14:26.245701 (+1671796us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400244ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.245829 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655204096}
I20260709 12:14:26.244813 27119 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.215381 27136 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.252244 27230 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.269256 27197 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.278789 27259 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:26.291568 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 492) took 2471 ms. Trace:
I20260709 12:14:26.315421 25173 rpcz_store.cc:276] 0709 12:14:23.820252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.824096 (+  3844us) service_pool.cc:224] Handling call
0709 12:14:24.722636 (+898540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.838487 (+115851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.287257 (+1448770us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54006a2550. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.289182 (+  1925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934567296}
I20260709 12:14:26.272300 27160 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.274304 27103 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.322057 27160 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.305644 27197 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:26.327826 27231 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:26.322264 27103 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.335835 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 264) took 1310 ms. Trace:
W20260709 12:14:26.337282 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.338294 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:26.336313 25174 rpcz_store.cc:276] 0709 12:14:25.025850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.025950 (+   100us) service_pool.cc:224] Handling call
0709 12:14:26.335573 (+1309623us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400734454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.335765 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669347712}
I20260709 12:14:26.340113 26748 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.353694 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:26.354792 26748 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.314934 27100 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.313694 27259 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:26.323256 26718 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.294911 27136 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.344249 27156 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.353075 27103 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.370543 26748 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.373618 27100 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.391407 27100 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.411726 27136 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.375047 27104 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.415055 26731 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.416182 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:26.417172 27104 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.374920 27267 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:26.428174 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 267) took 1054 ms. Trace:
I20260709 12:14:26.429764 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1108) took 2061 ms. Trace:
I20260709 12:14:26.430142 23852 rpcz_store.cc:276] 0709 12:14:24.368559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.780550 (+411991us) service_pool.cc:224] Handling call
0709 12:14:24.781027 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.781302 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:24.781328 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.287680 (+1506352us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b2c00000448. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:26.288090 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.374399 (+ 86309us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:26.374435 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.374508 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.400356 (+ 25848us) raft_consensus.cc:1685] finished
0709 12:14:26.400377 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.429699 (+ 29322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113010432,"thread_start_us":86024,"threads_started":1,"wal-append.queue_time_us":101026}
I20260709 12:14:26.433672 23852 raft_consensus.cc:1217] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Deduplicated request from leader. Original: 3.5->[5.6-7.7]   Dedup: 5.6->[7.7-7.7]
I20260709 12:14:26.434286 23852 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:14:26.382463 27103 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.437644 26718 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.428635 25177 rpcz_store.cc:276] 0709 12:14:25.374224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.568047 (+193823us) service_pool.cc:224] Handling call
0709 12:14:25.568391 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.568633 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.568660 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.248146 (+679486us) spinlock_profiling.cc:243] Waited 9.63 ms on lock 0x7b2c000e5e48. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:26.248614 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.315619 (+ 67005us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:26.315658 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.315736 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.420210 (+104474us) raft_consensus.cc:1685] finished
0709 12:14:26.420230 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.428088 (+  7858us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22150400,"thread_start_us":66676,"threads_started":1,"wal-append.queue_time_us":125433}
I20260709 12:14:26.441888 27220 raft_consensus.cc:993] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:26.429425 27104 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.438555 27103 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.440164 27267 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.455782 27158 raft_consensus.cc:1081] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:26.466519 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2703) took 2331 ms. Trace:
I20260709 12:14:26.466637 27111 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:26.466987 23851 rpcz_store.cc:276] 0709 12:14:24.135077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.555338 (+420261us) service_pool.cc:224] Handling call
0709 12:14:24.555610 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.555817 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:24.894893 (+339076us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.848922 (+954029us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b180003d720. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:26.465829 (+616907us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b180003d720. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a
0709 12:14:26.466139 (+   310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.466191 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:26.466242 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.466310 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.466438 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":413220,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2086,"spinlock_wait_cycles":51832320,"thread_start_us":415889,"threads_started":2,"wal-append.queue_time_us":76341}
I20260709 12:14:26.444301 27120 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.456275 27104 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.448112 27259 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.483925 23857 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:26.494040 26731 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.498345 26731 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.502413 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:26.503737 26731 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:26.514249 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.514294 27272 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:14:26.480393 27197 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.510993 26748 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.518657 27080 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:26.526070 27133 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.533357 26748 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.525679 27233 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.561902 27233 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.567783 26748 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.538952 27080 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.572022 27120 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.572913 27235 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:26.573493 27080 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.545408 27119 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.576161 27119 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.576390 27197 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.577452 23781 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:26.580924 27111 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:26.581413 27111 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.581467 23781 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:26.582679 23781 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:26.583091 23782 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:26.580912 23851 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:26.595358 23784 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:26.597064 23784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:26.597693 23784 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:26.597885 23780 pending_rounds.cc:85] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 6
I20260709 12:14:26.598356 23780 pending_rounds.cc:107] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:14:26.598592 23780 raft_consensus.cc:2889] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:26.604653 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.598075 23784 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.573330 27104 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.573356 27235 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:26.606030 27235 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.607057 27235 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.610919 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.611614 23853 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:26.607677 27004 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.621903 23853 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:26.623761 23781 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:26.587383 27267 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.626760 23784 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:26.638590 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.640868 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.649124 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "643746a7a11243df94746cf1081fc078"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:26.649856 23781 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:26.650213 23781 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.654862 23784 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.662254 27111 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.639657 27159 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:14:26.668295 27111 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.668648 27159 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:26.669193 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.669461 23784 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:26.655465 27197 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:14:26.679443 23855 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:26.681121 23856 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:26.681471 23855 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:26.682559 23856 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:26.682838 23855 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:26.664207 27262 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.433s	user 0.003s	sys 0.000s
I20260709 12:14:26.686205 23852 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:14:26.688004 23900 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:26.691228 27142 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.691829 27142 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.694980 27142 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.648546 27227 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:26.696071 27227 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.696460 27227 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:26.706183 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2431) took 1708 ms (client timeout 2171 ms). Trace:
W20260709 12:14:26.706688 23779 rpcz_store.cc:269] 0709 12:14:24.997605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.016283 (+ 18678us) service_pool.cc:224] Handling call
0709 12:14:26.705922 (+1689639us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400167954. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.706073 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492417792}
I20260709 12:14:26.708503 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.710093 27142 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:26.711755 27142 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:26.712380 27142 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:26.713354 27142 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.716569 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1114) took 1821 ms. Trace:
I20260709 12:14:26.717025 23859 rpcz_store.cc:276] 0709 12:14:24.894793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.897770 (+  2977us) service_pool.cc:224] Handling call
0709 12:14:26.716356 (+1818586us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400386654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.716496 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814817152}
I20260709 12:14:26.720099 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2445) took 1122 ms. Trace:
I20260709 12:14:26.726557 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.727876 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.728065 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2465) took 1010 ms. Trace:
I20260709 12:14:26.732342 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.720594 23783 rpcz_store.cc:276] 0709 12:14:25.597461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.918853 (+321392us) service_pool.cc:224] Handling call
0709 12:14:25.919363 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.944867 (+ 25504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.944901 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.257074 (+312173us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:14:26.464529 (+207455us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b2c00031b98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:26.465107 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.545166 (+ 80059us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:26.550687 (+  5521us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.550914 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.550984 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.657820 (+106836us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b180042c580. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:26.661027 (+  3207us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b180042c580. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1
0709 12:14:26.661450 (+   423us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b180042c580. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:26.719870 (+ 58420us) raft_consensus.cc:1685] finished
0709 12:14:26.719897 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.720035 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24847,"prepare.queue_time_us":5196,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":28544,"raft.run_cpu_time_us":3050,"raft.run_wall_time_us":303957,"spinlock_wait_cycles":136476416,"thread_start_us":79566,"threads_started":1,"wal-append.queue_time_us":102418}
I20260709 12:14:26.734532 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.734071 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.730166 23776 rpcz_store.cc:276] 0709 12:14:25.717563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.456067 (+738504us) service_pool.cc:224] Handling call
0709 12:14:26.456372 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.456708 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.456747 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.457204 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.553495 (+ 96291us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:26.722889 (+169394us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.727717 (+  4828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.727802 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.727850 (+    48us) raft_consensus.cc:1685] finished
0709 12:14:26.727863 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.728000 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":401,"prepare.queue_time_us":108118,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":2099,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":1826,"thread_start_us":245440,"threads_started":2,"wal-append.queue_time_us":85391}
I20260709 12:14:26.735105 23926 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.746457 23781 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.747956 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.752017 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.753151 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.754913 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.755868 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.669158 27159 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:26.757069 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:26.760689 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:26.737255 27254 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:26.736857 27167 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:26.769824 23925 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:26.767544 23778 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:26.786767 23781 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.737322 27039 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:14:26.766855 27243 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.763990 27267 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:14:26.817049 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 353) took 1825 ms. Trace:
I20260709 12:14:26.817534 23933 rpcz_store.cc:276] 0709 12:14:24.991695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.991841 (+   146us) service_pool.cc:224] Handling call
0709 12:14:25.010838 (+ 18997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.011238 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:26.433677 (+1422439us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18003065e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf 00007fb2ad7cc038
0709 12:14:26.540689 (+107012us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.758373 (+217684us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.763969 (+  5596us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.764108 (+   139us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:26.764143 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.764223 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.816749 (+ 52526us) raft_consensus.cc:1685] finished
0709 12:14:26.816851 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.816983 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46500,"prepare.queue_time_us":993685,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":31217,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":29162,"spinlock_wait_cycles":5869824,"thread_start_us":273498,"threads_started":2,"wal-append.queue_time_us":252821}
I20260709 12:14:26.769593 27134 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:14:26.765516 26981 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.810824 27068 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:26.811197 27224 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:26.812230 27034 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:26.793280 27120 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.813171 27156 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:26.813473 27238 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:26.769536 27167 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:26.769539 27104 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 12:14:26.839375 23825 leader_election.cc:336] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.534s (SENT)
I20260709 12:14:26.834189 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:26.806630 27254 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:14:26.845966 23777 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:26.856807 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:26.843319 27224 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:26.858747 27224 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.859956 27224 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.817330 27111 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:26.876581 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 470) took 1877 ms. Trace:
I20260709 12:14:26.877086 23777 rpcz_store.cc:276] 0709 12:14:24.987318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.989636 (+  2318us) service_pool.cc:224] Handling call
0709 12:14:26.844783 (+1855147us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540012cad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7a9a3d 00007fb2b2779cd0 00007fb2abcfe254 00007fb2abcfe1e2 00007fb2abcfe111 00007fb2abcfe08c 00007fb2abcfd342 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:26.857510 (+ 12727us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568815872}
I20260709 12:14:26.846622 26899 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:26.876546 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 200151us
I20260709 12:14:26.842007 27136 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 12:14:26.879443 27279 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:14:26.882894 25176 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:26.853148 27167 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:26.843793 27100 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:14:26.901402 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.902321 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.902567 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2466) took 1184 ms. Trace:
I20260709 12:14:26.903151 23858 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:26.903941 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.902989 23778 rpcz_store.cc:276] 0709 12:14:25.717950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.569348 (+851398us) service_pool.cc:224] Handling call
0709 12:14:26.570022 (+   674us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:26.570565 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.570594 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:26.571630 (+  1036us) log.cc:844] Serialized 582 byte log entry
0709 12:14:26.767397 (+195767us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:26.831187 (+ 63790us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.854674 (+ 23487us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.858224 (+  3550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.858308 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.902363 (+ 44055us) raft_consensus.cc:1685] finished
0709 12:14:26.902380 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.902510 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23356,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":5417,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":10773,"thread_start_us":195344,"threads_started":1,"wal-append.queue_time_us":261017}
W20260709 12:14:26.906889 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.906739 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.884605 26899 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:26.889072 27153 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.917243 27153 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.889638 27134 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:26.918702 27134 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:26.919734 27134 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.921048 27134 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:26.828528 27159 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.907440 26979 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.914762 26899 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.924679 26899 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.926535 26748 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:26.901191 27136 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:26.927174 26748 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:26.928158 27136 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:26.928568 27136 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:26.929607 26748 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:26.930379 27204 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:26.932734 27227 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:26.935328 25175 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:26.937150 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 497) took 1583 ms. Trace:
I20260709 12:14:26.937552 27227 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:26.938097 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.938772 23860 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:26.944203 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2441) took 1347 ms. Trace:
I20260709 12:14:26.945536 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:26.944679 23785 rpcz_store.cc:276] 0709 12:14:25.596505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.836942 (+240437us) service_pool.cc:224] Handling call
0709 12:14:25.837313 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.944707 (+107394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.944742 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.600524 (+655782us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b2c00031b98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:26.600973 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.752121 (+151148us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:26.752179 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.752259 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.943834 (+191575us) raft_consensus.cc:1685] finished
0709 12:14:26.943874 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.944131 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106958,"raft.queue_time_us":107090,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":15973,"spinlock_wait_cycles":3209984,"thread_start_us":150733,"threads_started":1,"wal-append.queue_time_us":205008}
I20260709 12:14:26.914450 23926 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:26.948376 23852 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:26.949460 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.949925 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.950562 23859 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.951296 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.952072 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.952347 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:26.952710 23856 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:26.950652 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:26.955364 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:26.955678 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:26.956410 23852 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:26.958461 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:26.959452 23860 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:26.960646 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:26.937482 25175 rpcz_store.cc:276] 0709 12:14:25.353719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.353880 (+   161us) service_pool.cc:224] Handling call
0709 12:14:25.354235 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.935044 (+1580809us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b540064d5a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad729dc0 00007fb2ad7b4d87 00007fb2ad7b5470 00007fb2ad7b1d8f 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c
0709 12:14:26.935769 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.936922 (+  1153us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540064cdd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:26.937086 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838024192}
I20260709 12:14:26.977634 23859 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:26.979573 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:26.984686 23930 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:26.992229 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:26.994529 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:26.985399 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 729860us
I20260709 12:14:26.984534 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.002296 23933 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:26.893440 27268 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.024849 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.026121 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:26.978310 27279 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:14:26.980216 27287 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:14:26.956729 26748 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:27.035193 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.978713 27227 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:26.972827 27185 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:27.039686 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.039091 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:26.983462 27225 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:27.041374 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.044055 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:27.032228 27080 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:27.035094 27021 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:27.034703 27233 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.034385 27090 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:27.079921 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.060127 27021 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:27.085435 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.083748 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 56740.3us
W20260709 12:14:27.087144 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.929720 27136 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.034966 27204 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:14:27.062098 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.089319 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:27.085717 27021 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.060313 27042 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.057511 27227 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:27.157910 27042 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.158161 27227 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.129107 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:27.161321 25174 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:27.161535 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72214.5us
I20260709 12:14:27.134142 27167 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.163544 27167 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.164433 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:27.165062 23927 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.165627 27068 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:27.166026 27068 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:27.166321 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.167048 23785 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:27.166819 27068 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.167528 26731 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:27.167940 26731 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:27.168236 23781 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:27.168560 23776 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 6. Ops: []
I20260709 12:14:27.135464 27136 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.168718 26731 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:27.169889 27068 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.170535 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:27.171625 23785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:27.172220 23785 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:27.172578 23785 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.173651 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:27.175136 23932 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:27.138070 27294 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.145368 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 483) took 1087 ms. Trace:
I20260709 12:14:27.059935 27080 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.081291 27185 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:14:27.089864 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.131933 25175 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:27.168092 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.188937 23777 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
W20260709 12:14:27.169019 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.126753 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.082363 27225 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:27.180405 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.193252 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.194126 25174 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:27.194340 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.182943 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.184226 27294 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.177583 23780 rpcz_store.cc:276] 0709 12:14:26.058114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.597341 (+539227us) service_pool.cc:224] Handling call
0709 12:14:26.597671 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:26.614005 (+ 16334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.614041 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:26.620706 (+  6665us) log.cc:844] Serialized 52 byte log entry
0709 12:14:26.832492 (+211786us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:26.832531 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.832600 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:27.145095 (+312495us) raft_consensus.cc:1685] finished
0709 12:14:27.145124 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:27.145300 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11072,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7711,"thread_start_us":183552,"threads_started":1,"wal-append.queue_time_us":279037}
I20260709 12:14:27.207775 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:27.209579 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.187068 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:27.188438 23784 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:27.189517 23778 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:14:27.210987 23778 pending_rounds.cc:85] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 8
I20260709 12:14:27.211261 23778 pending_rounds.cc:107] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:14:27.211498 23778 raft_consensus.cc:2889] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.191433 23779 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:27.195529 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.192198 23785 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.130923 27204 consensus_queue.cc:1243] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:14:27.198213 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:27.223417 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 197362us
I20260709 12:14:27.160887 27042 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.224248 23777 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.213327 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.185995 27230 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:27.169936 26731 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.190382 27185 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:14:27.283035 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.257318 23785 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:27.295500 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.094s (SENT)
W20260709 12:14:27.296643 23899 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.103s (SENT)
I20260709 12:14:27.187758 27080 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:27.356854 23784 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 7. Ops: []
W20260709 12:14:27.357434 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.357836 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:27.359051 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.198628 27100 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:27.360028 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.387027 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.198465 27268 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.368969 23783 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:27.223094 27267 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:27.394475 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.395166 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.402571 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.404968 23931 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:27.399999 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.849s (SENT) [suppressed 15 similar messages]
I20260709 12:14:27.226001 27167 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:27.410503 23932 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:27.413055 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.226159 25172 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:27.425699 23931 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:14:27.403079 27142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.022s	sys 0.023s
I20260709 12:14:27.197579 27111 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.253702 27230 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.429854 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.393558 27293 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.400866 27225 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.443533 27041 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:27.443763 27230 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.448827 27041 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.407850 27239 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:14:27.449622 27293 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.451378 27041 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.443799 27100 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:27.452006 27293 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:27.436975 27142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.023s	sys 0.024s
I20260709 12:14:27.463209 23927 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:27.465931 26955 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:27.466805 26955 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:27.470942 27230 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.473016 27294 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:27.481389 27041 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:27.482611 26641 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:27.483180 26641 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.485397 26641 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.425283 27224 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:27.427867 27167 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:27.492898 27167 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:27.486038 27134 consensus_meta.cc:287] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.003s
W20260709 12:14:27.495463 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.476269 27239 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:27.496179 27239 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:27.497108 27239 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.498512 27239 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.481986 27041 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.489033 27225 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.482645 27230 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.501652 27225 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.501644 27041 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.504948 27021 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:27.510243 27065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.009s	sys 0.025s
I20260709 12:14:27.516479 27080 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:27.519209 27065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.009s	sys 0.025s
W20260709 12:14:27.435768 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 222516us
I20260709 12:14:27.639268 27021 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:27.598070 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 175276us
I20260709 12:14:27.640574 27021 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.640884 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:27.641607 27021 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.530154 27034 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:27.634936 26748 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.532436 27080 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:27.645632 27156 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.645471 26748 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.646625 27156 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:27.610589 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.647398 27080 raft_consensus.cc:2764] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 12:14:27.630632 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 224132us
W20260709 12:14:27.630645 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 189888us
I20260709 12:14:27.558380 27153 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.609474 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:27.638128 26955 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:27.654040 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45032.3us
I20260709 12:14:27.653645 26955 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.551940 27159 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.644264 27124 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:27.656164 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.656859 26748 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.656567 27124 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:27.658910 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.644244 27034 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:27.661473 27034 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.635421 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 409) took 1439 ms. Trace:
I20260709 12:14:27.666028 27034 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.666028 23857 rpcz_store.cc:276] 0709 12:14:26.195950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.250978 (+ 55028us) service_pool.cc:224] Handling call
0709 12:14:27.634847 (+1383869us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54003cc654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:27.635156 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477988608}
I20260709 12:14:27.654053 23782 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:27.668946 26955 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:27.662974 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.674202 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.675446 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.552871 27158 raft_consensus.cc:993] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:27.543933 27194 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:27.669467 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2474) took 1926 ms. Trace:
I20260709 12:14:27.664749 27156 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:27.677728 27158 raft_consensus.cc:1081] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:27.669734 26955 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:14:27.635720 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 410) took 1290 ms. Trace:
I20260709 12:14:27.678241 23782 rpcz_store.cc:276] 0709 12:14:25.742821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.583904 (+841083us) service_pool.cc:224] Handling call
0709 12:14:27.653046 (+1069142us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:27.653561 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.669049 (+ 15488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.669282 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8989568}
I20260709 12:14:27.677807 27194 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:27.719092 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.719763 23776 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:27.720141 23782 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
I20260709 12:14:27.720940 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.722781 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:27.723795 23781 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.675313 23933 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:27.700573 27104 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:27.543496 27291 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.400s	user 0.007s	sys 0.004s
I20260709 12:14:27.697909 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.690519 23851 rpcz_store.cc:276] 0709 12:14:26.345626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.472741 (+127115us) service_pool.cc:224] Handling call
0709 12:14:26.473094 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.588303 (+115209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.628812 (+1040509us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003cc154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:27.635496 (+  6684us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247439744}
I20260709 12:14:27.701218 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:27.730876 23858 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:14:27.689519 27124 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:27.705037 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:27.541167 27259 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:27.564584 27254 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.001s	sys 0.011s
I20260709 12:14:27.708985 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:27.709586 25176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 500) took 2339 ms (client timeout 2171 ms). Trace:
I20260709 12:14:27.710464 27133 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:27.713106 27021 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.714133 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:27.754748 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.755755 25176 rpcz_store.cc:269] 0709 12:14:25.369686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.369817 (+   131us) service_pool.cc:224] Handling call
0709 12:14:27.709049 (+2339232us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400820354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:27.709220 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3445091200}
W20260709 12:14:27.865680 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.792164 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.764214 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.807915 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.896389 23780 raft_consensus.cc:2368] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:27.895066 23781 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:27.792318 23785 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:14:27.766819 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68581.3us
I20260709 12:14:27.905828 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2429) took 1161 ms. Trace:
I20260709 12:14:27.906375 23925 rpcz_store.cc:276] 0709 12:14:26.725836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.726301 (+   465us) service_pool.cc:224] Handling call
0709 12:14:27.887177 (+1160876us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b54002b6e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:27.887470 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32226560}
W20260709 12:14:27.907827 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.909929 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.791096 23824 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:27.907909 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:27.789383 27254 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.001s	sys 0.012s
I20260709 12:14:27.916070 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:27.917150 23776 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:27.919047 23777 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.508s	user 0.003s	sys 0.013s
I20260709 12:14:27.919637 23777 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:27.886788 27159 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:14:27.927965 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:27.929337 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 132928us
I20260709 12:14:27.931964 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.933018 23857 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:27.933564 27293 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:27.934468 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.934355 27293 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.938714 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:27.942031 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:27.866292 27153 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:14:27.944165 23785 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.791060 27158 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:27.954461 27158 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:27.955468 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.959635 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.961476 27268 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:27.962157 27268 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:27.964243 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1138) took 1018 ms. Trace:
I20260709 12:14:27.964839 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:27.965308 23776 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:27.964689 23858 rpcz_store.cc:276] 0709 12:14:26.945710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.951011 (+  5301us) service_pool.cc:224] Handling call
0709 12:14:27.730252 (+779241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.731272 (+  1020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.963954 (+232682us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400024250. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:27.964171 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783652096}
W20260709 12:14:27.967005 27322 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.069s	user 0.001s	sys 0.001s
I20260709 12:14:27.968334 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:27.866832 27224 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:27.969493 27197 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:27.969998 27224 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:27.971369 23783 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:27.972772 27224 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:27.973662 27268 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.799139 27156 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:27.976187 27156 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:27.973708 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.985741 23858 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:27.987448 23854 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:27.987890 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.794512 27194 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:27.990566 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.789028 27259 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:27.958186 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 167248us
I20260709 12:14:27.886056 27230 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:27.990563 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:27.961513 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:27.794246 27021 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.000329 27300 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:28.000972 23858 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:28.003461 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.008023 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:28.009088 25171 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:28.019338 23783 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:27.997753 27156 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.021150 23777 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:27.956113 27158 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.024209 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.001770 27300 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.030367 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:28.036314 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.038482 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.039669 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.040701 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.041370 23777 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:28.042390 23781 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:28.047868 23934 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:28.028326 27293 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.023415 27021 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.059937 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:28.061909 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.023203 27230 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.067925 23777 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:28.067350 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:28.074117 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.085031 26979 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.084203 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.085768 26979 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.086895 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.087805 23927 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:28.088177 26979 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.089454 27021 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.094265 26979 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.095099 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:28.102779 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.104028 23851 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:28.105327 26979 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:28.108206 27323 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:14:28.107719 26979 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.107852 27104 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.110985 23785 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:28.111259 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.110123 27104 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:28.111917 26979 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.113427 23781 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:28.103909 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 276) took 10954 ms. Trace:
I20260709 12:14:28.115924 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.114543 27224 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:28.104355 23776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:28.114578 27104 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.118233 23776 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:28.118662 23776 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.115821 23683 rpcz_store.cc:276] 0709 12:14:17.149638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.149925 (+   287us) service_pool.cc:224] Handling call
0709 12:14:28.103523 (+10953598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:24.971036 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:24.971633 (+   597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:24.971677 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:14:24.971698 (+    21us) tablet.cc:662] Decoding operations
0709 12:14:24.973175 (+  1477us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:24.973263 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:24.973291 (+    28us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:14:24.974487 (+  1196us) tablet.cc:701] Row locks acquired
0709 12:14:24.974512 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:14:24.974681 (+   169us) write_op.cc:270] Start()
0709 12:14:24.974812 (+   131us) write_op.cc:276] Timestamp: P: 1783599264974627 usec, L: 0
0709 12:14:24.974837 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:14:24.975516 (+   679us) log.cc:844] Serialized 5919 byte log entry
0709 12:14:25.472436 (+496920us) op_driver.cc:464] REPLICATION: finished
0709 12:14:25.716809 (+244373us) write_op.cc:301] APPLY: starting
0709 12:14:25.716920 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:25.717200 (+   280us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:25.717331 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:25.718849 (+  1518us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:25.719127 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 12:14:25.719170 (+    43us) write_op.cc:312] APPLY: finished
0709 12:14:25.719528 (+   358us) log.cc:844] Serialized 107 byte log entry
0709 12:14:25.778148 (+ 58620us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:25.794329 (+ 16181us) write_op.cc:454] Released schema lock
0709 12:14:25.804689 (+ 10360us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:25.959214 (+154525us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:25.964721 (+  5507us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27513,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":225889,"apply.run_cpu_time_us":4091,"apply.run_wall_time_us":178572,"num_ops":15,"prepare.queue_time_us":154238,"prepare.run_cpu_time_us":32722,"prepare.run_wall_time_us":217578,"raft.queue_time_us":128264,"raft.run_cpu_time_us":6172,"raft.run_wall_time_us":195937,"replication_time_us":448063,"thread_start_us":427652,"threads_started":4,"wal-append.queue_time_us":264464,"wal-append.run_cpu_time_us":45602,"wal-append.run_wall_time_us":1485259}]]}
I20260709 12:14:28.122619 27300 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.124256 27224 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.132371 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.135886 23857 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:28.136097 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:28.136474 27134 consensus_queue.cc:1059] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:28.137055 27124 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:28.140018 23934 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:28.142738 27194 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:28.143570 23681 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:28.145907 23681 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 8 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.150640 23927 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:28.144284 27248 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:28.155359 27248 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:28.156607 27248 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.160806 27248 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:28.156229 27224 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.166754 27134 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:28.167249 27134 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:28.168273 27134 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.169601 27134 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:28.174743 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.174472 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:28.177734 27295 consensus_queue.cc:1175] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "05679ef5e2b74861aad2a08b6a05e276" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 8 } }
I20260709 12:14:28.179499 25119 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.180595 27041 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:28.181425 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.181938 27041 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:28.182447 26979 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.183914 27234 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.183504 27039 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:28.186298 27039 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:28.180608 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.187513 27039 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.190852 27254 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.191565 27254 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.194063 27254 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.194904 26979 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:28.196043 27004 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:28.196904 26979 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:28.197302 26979 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:28.197468 27004 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:28.199291 26979 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.201982 23857 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:28.202174 26979 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.112815 27259 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:28.205276 26955 consensus_queue.cc:1175] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9e0bf56d58c5480a8d088c85566558da" responder_term: 5 status { last_received { term: 5 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 5 index: 7 } }
W20260709 12:14:28.211948 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 51831.5us
I20260709 12:14:28.134613 27197 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.203403 27041 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:28.217085 23776 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.220659 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.245790 23781 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:28.246767 23854 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:28.213909 27259 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:28.240458 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59113us
I20260709 12:14:28.219709 23681 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 7 to 10, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 10 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.259557 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.261826 27042 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:14:28.263163 23784 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:28.262928 23681 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:28.266387 23776 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.264707 23681 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:28.267427 27268 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:28.269152 27289 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.269994 27289 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.271849 27268 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:28.273447 27268 consensus_queue.cc:1243] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:28.274600 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.275418 23857 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:28.277556 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:28.278600 27295 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:28.279031 27295 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.279379 27295 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:28.280531 27230 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.282212 25171 raft_consensus.cc:1217] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Deduplicated request from leader. Original: 7.8->[8.9-8.9]   Dedup: 8.9->[]
I20260709 12:14:28.282831 25171 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:14:28.282698 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.283504 25172 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:28.284557 25176 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:14:28.284852 23824 connection.cc:256] RPC call timeout handler was delayed by -1.015672873000085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 12:14:28.286491 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.287387 25177 raft_consensus.cc:2410] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:28.287763 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.288450 23926 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:28.292290 27289 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.286542 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:28.240486 27039 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:28.309221 27133 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:28.311580 27133 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:28.312274 23927 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:28.312808 27133 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.224668 27104 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.296721 27254 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.329188 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.304806 27158 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:28.306552 23681 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:28.250288 27197 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.316016 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:28.348141 27197 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.348414 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:28.350412 25177 pending_rounds.cc:85] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 6
W20260709 12:14:28.350622 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 91169.3us
I20260709 12:14:28.351387 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.355408 25177 pending_rounds.cc:107] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:14:28.355788 25177 raft_consensus.cc:2889] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:28.355914 25174 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:28.356130 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:28.356338 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.357546 25119 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.350329 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:28.354151 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.360302 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.355873 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.367841 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:28.364421 27153 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:28.369254 27153 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.369187 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:28.372287 27153 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:28.372046 27268 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.377833 27268 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:28.354491 26955 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.378312 27268 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.378738 26955 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.381100 26955 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.343726 27234 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:14:28.382465 27289 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.382710 27234 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:28.354655 27254 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.383150 27289 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.384040 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.384814 23927 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:28.385581 27289 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.386437 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:28.387094 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.387490 27104 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.387917 27197 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:28.364773 27041 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:28.388115 27254 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.361179 25171 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:28.389219 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.390326 23854 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:28.391523 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:28.392274 27254 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.392699 27254 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.393126 27254 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.366823 27194 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:14:28.403453 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:28.403811 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:28.404255 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.404417 23781 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:28.408763 27325 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:14:28.383167 27234 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:28.396574 27197 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.402187 25172 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:28.412225 23681 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.415969 27197 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:28.447963 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 99671.5us
I20260709 12:14:28.433822 27042 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:28.453953 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:28.454165 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.410843 27234 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.454789 27042 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.481107 27234 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.444810 27153 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.482173 25176 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:28.492921 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.493790 23777 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:28.497615 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 129461us
I20260709 12:14:28.489672 23681 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 6 to 7, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
W20260709 12:14:28.510090 27328 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.142s	user 0.004s	sys 0.000s
I20260709 12:14:28.524354 23931 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:28.525662 27042 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.525727 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.526404 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.532128 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.532929 23934 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:28.533170 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.533881 23777 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:28.540597 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:28.499835 27248 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.555150 27248 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.556114 25172 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:28.549135 27156 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.551911 27295 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.586684 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:28.563048 27153 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.557516 27248 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.585907 27259 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:28.593050 23681 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 6, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.606503 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.609109 23854 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:28.619414 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.620224 23934 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:28.622206 23681 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 6, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 6 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.631336 23934 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:28.634330 25176 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:28.634426 27197 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.635572 27133 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:28.636032 27133 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:28.636394 27133 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:28.605394 27268 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.643679 27268 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.644194 27295 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.643217 23681 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:28.601466 27259 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.660351 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:28.662763 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51003.3us
I20260709 12:14:28.665539 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:28.666157 23851 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:28.671408 27342 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 12:14:28.672840 25173 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:28.663523 27259 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:28.682433 27197 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:28.674036 27336 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 12:14:28.680047 27289 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:14:28.685015 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.685416 27289 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.686057 27289 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.688539 27134 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:28.688992 27134 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.688836 27156 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.691496 27156 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.610440 27254 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.704133 27254 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:28.709944 23851 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:28.716195 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:28.716480 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.717613 25176 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:14:28.718528 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 185477us
I20260709 12:14:28.721195 27295 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:28.689384 27134 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.721791 27295 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:28.738627 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.721698 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:28.742511 23860 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.743567 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.739454 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.741477 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.745428 23934 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:28.713696 27289 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.747295 23932 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:28.749100 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 205176us
I20260709 12:14:28.746742 27224 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.791677 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:28.793857 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:28.806061 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 48760.7us
I20260709 12:14:28.808494 27295 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.813090 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.819785 23779 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:28.825806 23934 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:28.827162 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 328534us
W20260709 12:14:28.841324 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.171s (SENT)
I20260709 12:14:28.806910 27224 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:28.852236 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.861899 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:28.862991 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.863723 27341 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:14:28.865214 27224 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:28.860144 27104 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.867792 27104 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.869161 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.869778 25177 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.870158 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:28.870877 25176 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:28.863014 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:28.871749 23854 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:28.873071 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:28.862092 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.641s (SENT)
I20260709 12:14:28.876853 27259 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:28.876937 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:28.879029 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.834472 27004 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 12:14:28.885318 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 66304.7us
I20260709 12:14:28.869984 27104 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:28.895767 23854 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:28.877547 27134 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.890856 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:28.903278 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:28.907688 23749 connection.cc:256] RPC call timeout handler was delayed by -1.4879737219998788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:14:28.884938 27156 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.889675 27289 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.886952 27248 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:28.944491 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.956574 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.958600 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:28.958830 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 102142us
I20260709 12:14:28.959287 23934 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:28.960613 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.961366 23777 raft_consensus.cc:2368] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6.
I20260709 12:14:28.958175 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.917094 27104 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.915634 27224 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:28.961585 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.962836 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.968117 23926 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.967083 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.968534 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.968736 23778 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.969157 23777 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:28.918883 26955 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:28.972101 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.946130 27156 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:28.976332 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.962988 23931 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.976442 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.977802 23779 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:28.978282 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 105293us
I20260709 12:14:28.980655 25177 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:28.981863 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:28.982355 23784 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:28.986865 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.983104 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:28.987573 23781 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:28.971516 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.988044 23927 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:28.985529 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.992677 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.957337 27300 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.967636 27104 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.004999 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:28.963250 27230 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.007956 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:29.008611 27225 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.015383 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:28.983201 27248 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:29.018106 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.017903 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.018011 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:29.021283 23776 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:29.022655 23858 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:29.023677 23927 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.027987 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.012446 27104 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.026975 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.033303 23777 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:29.009042 27197 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:29.030336 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.030015 23932 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:29.032809 23784 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:29.042689 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:29.052006 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:29.052201 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:29.052709 23780 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.053113 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.053714 23783 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:29.054579 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:29.055764 27360 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:14:29.056448 27133 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.056975 27360 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:29.057370 27360 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:29.058238 27360 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:29.059333 27350 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:14:29.059754 27197 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.060956 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.061717 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.040709 27248 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.040913 27134 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.006322 27224 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.136822 23926 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:29.136888 23931 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:29.066459 23857 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:29.066491 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.060981 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:29.058328 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:29.024231 27259 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.074816 23778 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:29.137957 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 231581us
I20260709 12:14:29.124519 23860 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:29.125154 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50889.6us
I20260709 12:14:29.139540 23777 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:29.128144 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.064428 27133 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.121050 27360 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:29.141256 23853 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.142647 27347 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.149778 27347 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.138911 23929 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:29.023890 27230 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.155200 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.156047 23854 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:29.160841 23785 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:14:29.139413 27034 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.036725 27300 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.177793 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.178519 23860 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:29.082558 27225 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.180662 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:29.185794 23783 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:29.194784 23854 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:29.197116 23780 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:29.198319 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.197438 27366 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:14:29.212545 27366 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:29.212929 27366 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:29.213789 27366 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.215073 27366 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:29.216745 27357 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:14:29.218948 23825 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.719s (SENT)
I20260709 12:14:29.220716 27300 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.195888 27347 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:29.219394 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:29.266274 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62346.2us
I20260709 12:14:29.177043 27034 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.215044 27104 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.202761 27133 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:29.275805 23777 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:29.277812 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.278407 23860 raft_consensus.cc:2368] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:29.281826 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:29.282785 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.267534 27268 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:29.283479 23858 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 12:14:29.284363 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65477.1us
I20260709 12:14:29.287818 27378 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.288556 27378 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.290938 27225 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.291603 27230 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.293033 27224 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.281162 27156 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.294713 27197 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.280700 27365 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.295404 27259 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:29.289017 27368 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:29.299823 27361 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:29.299947 27134 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:29.300536 27134 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.301600 27034 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:29.302382 23825 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.728s (SENT)
I20260709 12:14:29.302778 27134 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:29.299448 27156 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.303784 27156 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.304214 27104 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.302811 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:29.306038 27156 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:29.306921 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.308867 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:29.309319 23899 leader_election.cc:336] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.907s (SENT)
I20260709 12:14:29.309791 23854 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:29.312994 23856 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:29.313689 23927 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:29.314029 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:29.325367 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 273280us
I20260709 12:14:29.290961 27378 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:29.315814 27104 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:29.328382 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.328759 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:29.329051 23851 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:29.295420 27373 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:14:29.314365 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.317059 27365 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.313694 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:29.331266 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:29.300472 27361 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.332437 27365 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:29.325996 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.326777 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:29.309787 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:29.314816 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:29.331836 25177 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:29.317621 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.333742 23853 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:29.327898 27248 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:29.337261 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.344666 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:29.345243 27361 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.347258 27156 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.348150 27295 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:29.349272 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:29.352807 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.353628 23853 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:29.297431 27259 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:29.359396 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:29.362411 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:29.352294 27346 raft_consensus.cc:2764] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:29.365746 27370 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.366406 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.366365 27370 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.367277 23854 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:29.368700 27370 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:29.371511 27362 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:29.372773 27259 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.348826 27289 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:29.375890 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66627.4us
I20260709 12:14:29.376387 27289 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.381134 23853 raft_consensus.cc:2410] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:29.360455 27295 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:29.382908 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.358088 27124 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.385063 23851 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:14:29.376022 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 44976.9us
I20260709 12:14:29.373526 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.348232 27156 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.386525 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.388854 23859 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:29.396942 27295 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:29.397971 27295 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.386977 27344 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:29.401266 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.404551 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:29.405771 27034 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:29.406220 27034 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.406615 27034 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.407925 27362 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:29.408614 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:29.408995 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 127941us
I20260709 12:14:29.409112 27362 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.395206 27124 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.412336 27042 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.412463 27124 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.382480 27370 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:29.377624 27254 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:14:29.417182 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.420526 23853 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:29.414417 27156 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:29.428151 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 99503us
I20260709 12:14:29.438235 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:29.439189 27267 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:29.450892 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:29.454891 23926 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:29.459695 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:29.461501 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:29.462193 27289 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:29.466642 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:29.462234 23927 raft_consensus.cc:2368] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:29.472890 23777 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:29.472955 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.473491 23777 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:29.473843 23853 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:29.474022 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.474663 23860 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:29.474411 23777 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.475520 23777 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:29.474411 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:29.481428 23854 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:29.485532 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:29.485865 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45627.8us
I20260709 12:14:29.436826 27254 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:29.474573 27370 raft_consensus.cc:2764] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:29.485126 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.512612 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:29.514088 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.515275 23859 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:14:29.439627 27267 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:29.515442 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:29.520159 23777 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:14:29.519659 23751 leader_election.cc:400] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:29.520697 23751 leader_election.cc:403] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:29.520560 23777 pending_rounds.cc:85] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 6
I20260709 12:14:29.521238 23777 pending_rounds.cc:107] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:14:29.521481 23777 raft_consensus.cc:2889] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:29.522532 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.522758 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.517872 23925 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:29.528659 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.523167 23856 raft_consensus.cc:2376] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9. Re-sending same reply.
I20260709 12:14:29.517941 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:29.530521 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.529202 23860 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:29.492038 27134 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:29.539368 23925 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:29.528429 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:29.467811 27369 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 12:14:29.561645 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 184672us
I20260709 12:14:29.512053 27254 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:29.559793 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68343.5us
I20260709 12:14:29.558956 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:29.525087 27267 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.563130 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:29.566491 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88546.7us
I20260709 12:14:29.443734 27034 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.569191 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:29.489540 27378 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:14:29.492182 27156 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:14:29.569816 23925 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:29.574165 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 81367.3us
I20260709 12:14:29.534612 27124 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:29.541401 27134 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.538308 27368 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:29.583946 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:29.564461 27300 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:29.560314 27042 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:29.594261 23932 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:29.592517 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:29.570001 27194 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:29.598414 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:29.579394 27124 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.589310 27134 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:29.588231 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:29.566290 27345 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:14:29.615875 27369 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:29.616459 27194 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.623428 27124 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:29.648486 27382 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:14:29.649366 27381 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:14:29.654318 23925 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:29.655371 27254 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:29.628801 27368 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:29.619616 27289 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:29.664996 27300 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:29.665710 27224 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:29.667819 27392 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:29.638188 27369 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:29.681679 23824 leader_election.cc:400] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:29.712639 23824 leader_election.cc:403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:14:29.683578 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:29.713583 27289 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.714869 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:29.648377 27380 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:14:29.645288 27194 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:29.688714 27368 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.609174 27230 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:14:29.684562 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49646.8us
I20260709 12:14:29.637841 25173 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:29.726027 23932 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:29.735531 25176 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:29.740638 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:29.742017 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:29.743845 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.738940 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.700407 27224 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:29.749447 27339 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:29.753197 27339 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:29.717437 27371 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:29.748795 23851 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.721561 27300 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:29.721218 27365 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:29.726740 27254 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:29.718482 27392 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:29.758236 27348 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:14:29.738524 27368 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:29.764367 27366 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:29.979346 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 361237us
I20260709 12:14:29.765578 27365 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:29.847558 27382 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:14:29.941128 27391 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:14:29.855778 27369 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.031833 27392 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.034428 27370 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:14:30.100180 23825 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.849s (SENT)
I20260709 12:14:30.104020 23851 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:29.686177 27346 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:14:30.029532 27300 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:30.114934 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 364427us
W20260709 12:14:30.121186 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 437772us
I20260709 12:14:30.129454 27034 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:30.134007 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:30.135413 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:30.036543 27224 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.113598 27365 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:30.131753 25171 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:14:30.150130 27267 consensus_meta.cc:287] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.003s
I20260709 12:14:30.123188 27370 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:30.156167 27388 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:14:30.113834 27124 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.157637 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:30.158331 27361 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.158552 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2494) took 1146 ms. Trace:
I20260709 12:14:30.159682 27289 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:30.159874 27369 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.228677 23781 rpcz_store.cc:276] 0709 12:14:29.011859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.012632 (+   773us) service_pool.cc:224] Handling call
0709 12:14:29.022422 (+  9790us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:29.195280 (+172858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.195312 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:29.268755 (+ 73443us) log.cc:844] Serialized 52 byte log entry
0709 12:14:29.438168 (+169413us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:29.438819 (+   651us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.439642 (+   823us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.439850 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.439924 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.158350 (+718426us) raft_consensus.cc:1685] finished
0709 12:14:30.158372 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.158494 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105,"prepare.queue_time_us":287,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":237047,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":26426,"thread_start_us":246927,"threads_started":2,"wal-append.queue_time_us":242224}
I20260709 12:14:30.160276 27371 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:30.162010 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2498) took 1111 ms. Trace:
I20260709 12:14:30.162770 27339 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:30.164633 27134 consensus_meta.cc:287] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.528s	user 0.017s	sys 0.000s
W20260709 12:14:30.164965 23899 leader_election.cc:336] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.058s (SENT)
I20260709 12:14:30.151304 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:30.144626 27230 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 12:14:30.181181 27267 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.285310 23782 rpcz_store.cc:276] 0709 12:14:29.050713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.053323 (+  2610us) service_pool.cc:224] Handling call
0709 12:14:29.053791 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:29.182565 (+128774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.182596 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:29.252261 (+ 69665us) log.cc:844] Serialized 52 byte log entry
0709 12:14:29.515072 (+262811us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b18003886c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:14:29.580033 (+ 64961us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:29.608037 (+ 28004us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.610502 (+  2465us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.610692 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.610788 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.112500 (+501712us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b18003886c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f
0709 12:14:30.161770 (+ 49270us) raft_consensus.cc:1685] finished
0709 12:14:30.161797 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.161942 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64469,"prepare.queue_time_us":50734,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":1899,"raft.queue_time_us":233047,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":22081,"spinlock_wait_cycles":27389184,"thread_start_us":175860,"threads_started":2,"wal-append.queue_time_us":444612}
I20260709 12:14:30.306818 27230 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:30.307792 27230 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:30.308178 27230 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:30.308530 27230 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:30.313236 27267 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:30.316033 27370 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.317875 27370 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.318553 27230 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.327262 27230 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:30.328225 27374 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.335808 27374 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.338017 27374 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:30.281948 27104 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:30.354346 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:30.293627 27259 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.297060 27403 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:30.363428 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:30.252693 27361 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:14:30.495849 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 132506us
I20260709 12:14:30.303834 27134 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.496884 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:30.531617 27358 raft_consensus.cc:2764] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:30.532541 27408 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.533107 27408 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.534119 27401 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:30.535249 27408 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:30.529862 27104 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:30.538460 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 390924us
W20260709 12:14:30.497148 27194 consensus_meta.cc:287] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.752s	user 0.011s	sys 0.015s
I20260709 12:14:30.444903 27259 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:30.551838 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.533123 27361 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:14:30.564257 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 318539us
W20260709 12:14:30.564669 23899 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.188s (SENT)
I20260709 12:14:30.567054 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:30.569391 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.555505 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 439) took 1198 ms. Trace:
I20260709 12:14:30.559638 27104 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:30.550979 27300 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:30.560511 27366 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:30.560039 27194 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.610523 27124 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:30.570181 27365 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:30.639921 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 488793us
I20260709 12:14:30.584524 23858 rpcz_store.cc:276] 0709 12:14:29.356670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.356788 (+   118us) service_pool.cc:224] Handling call
0709 12:14:29.364390 (+  7602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.384317 (+ 19927us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:29.399890 (+ 15573us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.457785 (+ 57895us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b18003b86c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:29.510290 (+ 52505us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b18003b86c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:29.762464 (+252174us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.764397 (+  1933us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.884468 (+120071us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.028591 (+144123us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:30.028871 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.555117 (+526246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.555202 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.555280 (+    78us) raft_consensus.cc:1685] finished
0709 12:14:30.555294 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.555441 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90610,"prepare.queue_time_us":12121,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":111,"raft.queue_time_us":159439,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":4594,"spinlock_wait_cycles":93003264,"thread_start_us":666328,"threads_started":2,"wal-append.queue_time_us":444976}
W20260709 12:14:30.624109 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54382.4us
I20260709 12:14:30.859053 27124 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:30.861465 27397 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.698s	user 0.002s	sys 0.000s
I20260709 12:14:30.874289 27361 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:30.874871 27104 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.973361 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2455) took 1522 ms. Trace:
I20260709 12:14:30.973809 23930 rpcz_store.cc:276] 0709 12:14:29.451432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.451739 (+   307us) service_pool.cc:224] Handling call
0709 12:14:29.460586 (+  8847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.487975 (+ 27389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.488006 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.488608 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:14:29.765063 (+276455us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:29.765106 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.765179 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.860721 (+1095542us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:30.973046 (+112325us) raft_consensus.cc:1685] finished
0709 12:14:30.973068 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.973292 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28174,"raft.queue_time_us":30805,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15619,"spinlock_wait_cycles":5323008,"thread_start_us":276064,"threads_started":1,"wal-append.queue_time_us":658331}
I20260709 12:14:30.876185 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:30.975906 27345 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.976207 25171 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:30.976487 27345 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.877067 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:30.879937 27225 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.880753 27372 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.984022 27372 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.986299 27372 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:30.881407 27378 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:30.876240 27366 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.882520 27405 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.995967 27405 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.998167 27405 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:30.998124 27194 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:30.882931 27254 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:30.883412 27411 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.392s	user 0.015s	sys 0.004s
W20260709 12:14:30.890843 27399 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.854s	user 0.005s	sys 0.018s
I20260709 12:14:31.000501 27254 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.988600 27041 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.002207 27224 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:30.902994 27346 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:31.003069 27378 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.003329 27346 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:30.911347 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:31.004053 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 92820.7us
I20260709 12:14:30.913928 27158 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:30.631219 27004 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 12:14:30.895705 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.915623 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 350385us
W20260709 12:14:30.916038 27392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.000s	sys 0.005s
I20260709 12:14:30.916801 27413 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:31.016057 27378 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:30.918519 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:30.918933 27414 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.016443 27418 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:31.017230 27414 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.017323 27418 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.018129 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:31.018697 23855 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:30.919654 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:30.922040 27356 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:30.928568 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2461) took 1389 ms. Trace:
I20260709 12:14:31.019551 27418 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.019862 23854 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:31.024191 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:31.016503 23860 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:14:30.938376 27402 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.785s	user 0.002s	sys 0.010s
W20260709 12:14:30.943703 27394 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.061s	user 0.001s	sys 0.005s
W20260709 12:14:30.944833 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 915484us
W20260709 12:14:30.950942 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 793340us
I20260709 12:14:31.061441 23855 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:30.999939 27369 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.730s	user 0.016s	sys 0.000s
W20260709 12:14:30.952934 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.963462 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:30.951465 27348 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:30.983270 25173 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:30.979959 27225 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:30.979941 27124 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:30.995162 27365 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:30.896970 27295 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.002336 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2788) took 1262 ms. Trace:
I20260709 12:14:31.002707 27254 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:30.952196 27368 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.836s	user 0.006s	sys 0.011s
I20260709 12:14:31.015363 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "102493ff11494b009f79963bb57be511"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:14:31.015671 27392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.000s	sys 0.005s
I20260709 12:14:31.012642 27289 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:31.022583 23851 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:31.021513 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.024798 23858 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:30.979445 27345 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.067477 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:31.068737 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:31.071530 23825 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.597s (SENT)
W20260709 12:14:31.074868 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 510169us
I20260709 12:14:31.066833 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.016793 27358 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:31.019469 27414 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.003194 27041 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:31.167300 23899 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.296s (SENT)
I20260709 12:14:31.148821 23934 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:30.954656 27300 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.149140 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.150293 23854 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:31.167799 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.166139 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.020241 23931 rpcz_store.cc:276] 0709 12:14:29.539474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.563556 (+ 24082us) service_pool.cc:224] Handling call
0709 12:14:29.563848 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.564066 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:29.622270 (+ 58204us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.927980 (+1305710us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.928218 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.928270 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:30.928312 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.928373 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.928481 (+   108us) inbound_call.cc:177] Queueing success response
0709 12:14:30.941161 (+ 12680us) spinlock_profiling.cc:243] Waited 9.78 ms on lock 0x7b240021ebe8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa54b9c4 00007fb2aa549acb 00007fb2aa704dc9 00007fb2aa704a55 00007fb2aa70409d 00007fb2ad33610e 00007fb2ad34d88c 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221
Metrics: {"mutex_wait_us":826078,"prepare.queue_time_us":958619,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":451269,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":236,"spinlock_wait_cycles":22482688,"thread_start_us":530370,"threads_started":3,"wal-append.queue_time_us":526703}
I20260709 12:14:31.079715 23933 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:31.121888 23898 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.168671 23852 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:31.012216 27400 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.181599 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2456) took 1699 ms. Trace:
I20260709 12:14:31.182657 25177 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
W20260709 12:14:31.069748 27370 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.751s	user 0.017s	sys 0.000s
I20260709 12:14:31.172204 23856 rpcz_store.cc:276] 0709 12:14:29.739947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.740071 (+   124us) service_pool.cc:224] Handling call
0709 12:14:29.745329 (+  5258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.753051 (+  7722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.753082 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.753619 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.166531 (+412912us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:30.166572 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.166644 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:31.002126 (+835482us) raft_consensus.cc:1685] finished
0709 12:14:31.002151 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.002278 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4144,"raft.queue_time_us":8479,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":10414,"thread_start_us":412537,"threads_started":1,"wal-append.queue_time_us":546795}
I20260709 12:14:31.181993 23927 rpcz_store.cc:276] 0709 12:14:29.482396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.482532 (+   136us) service_pool.cc:224] Handling call
0709 12:14:29.491917 (+  9385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.492143 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:29.593172 (+101029us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.890154 (+1296982us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:30.929620 (+ 39466us) log.cc:844] Serialized 14 byte log entry
0709 12:14:31.073272 (+143652us) log.cc:844] Serialized 14 byte log entry
0709 12:14:31.169692 (+ 96420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.169746 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:31.169796 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.169857 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.181533 (+ 11676us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":993523,"prepare.queue_time_us":1044917,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":116,"raft.queue_time_us":330468,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":246,"spinlock_wait_cycles":163952896,"thread_start_us":561640,"threads_started":3,"wal-append.queue_time_us":427834}
I20260709 12:14:31.022298 27371 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.146s
I20260709 12:14:31.188550 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.073163 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:31.183825 25169 raft_consensus.cc:606] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:31.173971 27358 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.220429 25169 raft_consensus.cc:614] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:31.194553 27406 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:31.221976 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 153289us
I20260709 12:14:31.222259 27406 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.223901 27421 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.224326 27426 raft_consensus.cc:2764] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:14:31.224484 27421 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.220227 27369 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.191567 27194 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.195376 27419 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.190541 27400 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:31.196316 27420 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
W20260709 12:14:31.197131 27410 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.654s	user 0.008s	sys 0.003s
W20260709 12:14:31.197340 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 126315us
W20260709 12:14:31.250842 23825 connection.cc:256] RPC call timeout handler was delayed by -1.1874323689999073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19210000000000002s for a response
W20260709 12:14:31.251863 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.670s (SENT)
I20260709 12:14:31.252261 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.197646 27414 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:31.184265 27254 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:31.207890 27415 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.117204 27259 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:31.226486 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 104685us
I20260709 12:14:31.235479 25172 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:14:31.181205 27295 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.119083 27425 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.120493 27225 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.024380 27158 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.235522 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.020128 27356 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.016808 27413 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.212025 27134 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:31.219348 27345 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:31.227617 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.277992 23781 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:31.238466 27289 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:31.180804 27368 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:31.289850 23825 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.679s (SENT)
W20260709 12:14:31.290786 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 269122us
I20260709 12:14:31.291755 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.236693 27369 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.309401 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.283273 27156 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:31.307490 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.276573 27412 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.310716 23934 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:31.275820 27417 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:31.312096 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76659us
I20260709 12:14:31.112867 27346 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.282387 27361 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:31.286146 27365 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:31.276189 27401 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:14:31.250332 27194 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.320654 23784 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:31.330219 23777 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:31.337181 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.330823 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:31.342813 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52995.1us
W20260709 12:14:31.350204 23825 connection.cc:256] RPC call timeout handler was delayed by -1.1041396559999157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:14:31.350543 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0650323689999368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
I20260709 12:14:31.078203 27348 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:31.317083 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 168022us
I20260709 12:14:31.283830 27254 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.321789 27156 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.327361 27416 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:31.322072 27134 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.348577 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 121046us
I20260709 12:14:31.356894 23781 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:14:31.357016 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0304681169998275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:14:31.241367 27419 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.352365 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:31.375300 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.341547 25171 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
W20260709 12:14:31.370450 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61707.2us
I20260709 12:14:31.285615 27415 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:31.378000 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.348784 27346 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:31.295809 27425 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.344681 27417 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:31.311811 27345 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.356956 27427 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.356824 27225 raft_consensus.cc:2764] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:31.359680 27156 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:31.385875 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.403748 23855 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:31.401609 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:31.386191 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55451.3us
I20260709 12:14:31.353502 27434 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:31.359300 27416 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.386601 27370 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:31.390599 27421 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.390630 27369 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.383994 25169 raft_consensus.cc:606] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.391314 27440 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:31.392020 27412 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.395888 27415 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.396718 27041 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:31.399166 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.402809 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:31.471741 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 89068.6us
I20260709 12:14:31.404594 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.353978 27361 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.490062 23855 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:31.356189 27289 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.374681 27134 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.492764 23854 raft_consensus.cc:2368] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10.
I20260709 12:14:31.496232 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.510586 23777 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:31.488644 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.511413 25176 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:31.511294 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.515012 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:31.397540 27365 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.516856 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.517987 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.517517 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.505941 23856 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:31.523325 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.525856 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.390676 27124 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.509913 27416 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:31.485775 25169 raft_consensus.cc:614] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:31.528995 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.466363 27158 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.466516 27406 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.542706 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.500428 27368 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.464062 27413 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.465425 27194 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.488020 27224 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:31.461313 27356 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.555899 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.556689 23781 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:31.526665 25176 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:31.391347 27358 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.544044 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.534359 23858 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:31.563576 23851 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:31.496275 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.544826 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:31.570887 23856 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:31.540755 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.571385 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.543326 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.546506 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:31.497180 27419 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.502074 27254 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.513027 27295 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:31.504168 27440 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:31.553918 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.559089 27194 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.559027 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.559756 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.560317 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:31.559321 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.565886 25172 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:31.583464 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.568980 25177 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:31.572109 23932 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:31.546072 27124 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:31.575202 23926 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:31.542261 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.425576 27427 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.536262 27365 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.580448 25169 raft_consensus.cc:606] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:31.615801 25169 raft_consensus.cc:614] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:31.590377 27194 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.592168 27134 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:31.593190 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.594197 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.592172 27295 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.599291 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:31.605742 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.563745 27439 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:31.472359 27417 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.611035 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:31.590749 27432 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:31.617424 23854 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:31.503908 27412 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.624279 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:31.578969 27224 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:31.628687 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.629971 27194 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.630328 27440 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:31.636658 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 247144us
I20260709 12:14:31.642593 23854 pending_rounds.cc:85] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 8
I20260709 12:14:31.647011 23854 pending_rounds.cc:107] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:14:31.647307 23854 raft_consensus.cc:2889] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:31.608109 27124 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.645337 23781 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:14:31.609957 27348 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.648874 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.656531 23926 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:31.653374 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.657521 25172 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:31.653012 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.629854 27427 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.664284 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:31.632817 27368 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:31.666600 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.643990 27158 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:31.664052 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:31.646302 27419 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.670171 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.667336 23930 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:31.672606 23932 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:31.655386 27124 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:31.682435 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.685011 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.686075 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.688745 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:31.691888 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.692495 23749 connection.cc:256] RPC call timeout handler was delayed by -2.1535287840001729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 12:14:31.693055 23749 connection.cc:256] RPC call timeout handler was delayed by -2.0815287840000565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 12:14:31.694064 23749 connection.cc:256] RPC call timeout handler was delayed by -1.9717287840001063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 12:14:31.694521 23749 connection.cc:256] RPC call timeout handler was delayed by -1.9357287840000481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 12:14:31.694875 23749 connection.cc:256] RPC call timeout handler was delayed by -1.9195287840000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 12:14:31.695325 23749 connection.cc:256] RPC call timeout handler was delayed by -1.8331287840001096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:14:31.695696 23749 connection.cc:256] RPC call timeout handler was delayed by -1.8304287840001052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 12:14:31.696039 23749 connection.cc:256] RPC call timeout handler was delayed by -1.7953287840000485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 12:14:31.696384 23749 connection.cc:256] RPC call timeout handler was delayed by -1.7701287840002351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 12:14:31.696449 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:31.696884 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5982287840001845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:14:31.697232 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5946287840001787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260709 12:14:31.697863 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.699229 27224 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.702651 27361 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:31.676262 27368 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.703785 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.704949 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.705564 27368 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.705962 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.706975 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.707410 27365 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.708113 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.708714 23930 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:31.709115 27412 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:31.695843 23825 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.766s (SENT)
I20260709 12:14:31.710314 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.679229 27346 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:31.684320 27419 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.713757 27419 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:31.691695 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.707525 23926 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:31.716965 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.709560 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.718302 23930 raft_consensus.cc:2410] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:14:31.719262 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.719554 27124 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.720196 23926 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:31.719873 23930 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:31.676004 27158 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.725276 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.720578 27156 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.679320 27370 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:31.697551 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5739287840001452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 12:14:31.733086 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5703287840001394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
W20260709 12:14:31.733505 23749 connection.cc:256] RPC call timeout handler was delayed by -1.5388287840000885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 12:14:31.733860 23749 connection.cc:256] RPC call timeout handler was delayed by -1.4587287840001864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23070000000000002s for a response
W20260709 12:14:31.734153 23749 connection.cc:256] RPC call timeout handler was delayed by -1.1104287840000779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26940000000000003s for a response
I20260709 12:14:31.648593 27439 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.730279 27158 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:31.741670 23934 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:31.733258 23824 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.744709 27358 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:31.745126 27358 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:31.745465 27358 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:31.741103 27415 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:31.746201 27370 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:31.747254 27370 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.742620 27346 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:31.749708 27346 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:31.742267 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f333e0a484248ce88f0905e18500838"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:14:31.751798 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55977.5us
I20260709 12:14:31.764353 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.766700 27156 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.771508 27439 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:31.739751 27365 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:31.772735 27365 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:31.734957 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.791567 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:31.801384 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.802165 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:31.802217 23860 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:31.808658 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.773730 27365 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.827636 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.833838 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:31.858140 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 257593us
W20260709 12:14:31.858963 23899 leader_election.cc:336] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.914s (SENT)
I20260709 12:14:31.859377 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:31.860368 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:31.862700 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:31.875825 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48266.6us
I20260709 12:14:31.755028 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:31.884853 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:31.883960 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:31.890144 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 224518us
I20260709 12:14:31.860618 27225 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.866557 27392 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:31.860291 27413 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:31.821084 25169 raft_consensus.cc:606] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:31.865322 27439 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:31.874228 27438 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.899984 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:31.879994 27348 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:31.880095 27406 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:31.903360 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 166862us
I20260709 12:14:31.909209 23853 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:31.896162 27361 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:31.941795 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:31.936331 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.907290 25114 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:31.891633 27124 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:31.908689 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.868s (SENT)
I20260709 12:14:31.936010 27440 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.938648 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 186214us
W20260709 12:14:31.981174 23825 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.854s (SENT)
I20260709 12:14:31.981629 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:31.902515 27392 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.895982 25173 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:31.964772 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.940717 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:31.902037 27225 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:31.969753 27423 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:31.976691 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:31.982662 23853 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:31.942334 27361 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.990613 23930 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:31.898993 27416 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:32.001063 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:32.004843 23825 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.860s (SENT)
W20260709 12:14:32.004966 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68608.7us
I20260709 12:14:31.990391 27392 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:31.987969 27370 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.001554 27417 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.004184 27361 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.002952 27438 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:32.018369 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.860s (SENT)
I20260709 12:14:32.018883 23825 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:32.019973 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.017103 27370 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.030402 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:32.030896 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.032399 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 352373us
I20260709 12:14:32.008543 27194 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:32.000458 27440 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.007851 27406 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.015678 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:32.047549 27194 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:32.062728 27194 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.047675 27369 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.064247 27369 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.066424 27369 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:32.066802 27158 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.069316 27158 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:32.048698 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:32.075114 25169 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:32.009192 27224 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:32.044265 23930 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:32.045833 25175 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:14:32.046012 27427 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:32.046744 27450 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.047174 27370 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:32.058681 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 150035us
I20260709 12:14:32.041935 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:32.059301 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 53612.7us
I20260709 12:14:32.096825 23856 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:32.232259 27425 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:32.265060 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.266571 27401 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.262082 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 243732us
I20260709 12:14:32.267110 27401 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.267518 27417 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:32.266039 27450 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.097461 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:32.104207 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 208713us
I20260709 12:14:32.269191 27443 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:14:32.209755 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.269860 27443 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:32.270385 27369 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:32.270880 27443 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:32.058929 27444 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:14:32.272467 27443 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:32.154192 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 152137us
W20260709 12:14:32.273294 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.135269 27454 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.061s	user 0.000s	sys 0.004s
W20260709 12:14:32.273063 23825 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.950s (SENT)
I20260709 12:14:32.078775 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:32.276669 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 179293us
I20260709 12:14:32.277135 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.277593 23852 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.059186 27225 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:32.058017 27447 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:14:32.051858 25171 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:32.049417 27371 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.284991 25114 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:32.285959 23899 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.721s (SENT)
I20260709 12:14:32.054929 27416 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:32.287505 23898 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:32.290735 25119 leader_election.cc:400] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10.
I20260709 12:14:32.291152 25119 leader_election.cc:403] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:32.269186 27450 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.288128 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.276413 27346 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.273494 27224 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:32.291819 27427 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:32.296825 27156 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.296710 25172 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:32.302839 23852 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:14:32.295885 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.246s (SENT)
I20260709 12:14:32.283564 27425 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.056802 27358 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.053974 27361 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.329221 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.329861 23900 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:32.330479 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44546.1us
W20260709 12:14:32.343025 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 55576us
I20260709 12:14:32.345347 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.345602 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.342020 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1174) took 1321 ms. Trace:
I20260709 12:14:32.282406 27254 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:14:32.346439 23859 rpcz_store.cc:276] 0709 12:14:31.020884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.023035 (+  2151us) service_pool.cc:224] Handling call
0709 12:14:31.023310 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.378007 (+354697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.378039 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.378080 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:31.378113 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.341744 (+963631us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:32.341794 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.341950 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815253248}
I20260709 12:14:32.289323 27365 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:32.348526 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:32.328003 27156 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.351338 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.325316 27224 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:32.351627 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.353955 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.354588 23853 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.355844 23856 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:32.364009 23853 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:32.342536 27425 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.285745 27452 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:14:32.369192 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.370052 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.328161 27405 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.366370 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:32.347541 27430 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:32.393575 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 97727us
I20260709 12:14:32.327643 27346 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.354995 27417 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.384169 27156 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.400645 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:32.386983 27425 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.355870 27004 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:32.404762 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 139778us
I20260709 12:14:32.367194 27469 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.407727 27370 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:32.408278 27346 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:32.413977 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:32.418367 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 123215us
I20260709 12:14:32.419622 25114 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:32.406448 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.420849 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:32.421537 23930 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:32.420975 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:32.346433 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55972.3us
W20260709 12:14:32.424220 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.075s (SENT) [suppressed 38 similar messages]
I20260709 12:14:32.396135 27401 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.404184 27417 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.427187 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.427067 27413 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:32.428272 27392 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.427430 27346 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:32.429474 27346 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:32.429853 27346 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:32.432159 23825 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.130s (SENT)
I20260709 12:14:32.432931 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.408401 27004 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.368117 27467 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:14:32.428857 27413 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.406018 27474 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10.
I20260709 12:14:32.450045 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.408293 27370 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:32.412989 27400 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:32.427476 27417 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.463251 27430 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:32.468918 27430 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:14:32.479198 27465 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:14:32.481997 27474 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.447547 27004 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.428963 27434 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 12:14:32.496068 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46100.2us
I20260709 12:14:32.419790 27427 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.499812 27361 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:32.500995 27470 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:14:32.453605 27471 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.473302 27392 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.504541 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.515798 27370 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.483950 27400 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.493310 27474 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:32.532598 25174 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 800) took 1016 ms. Trace:
I20260709 12:14:32.546443 25174 rpcz_store.cc:276] 0709 12:14:31.515978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.516101 (+   123us) service_pool.cc:224] Handling call
0709 12:14:32.531805 (+1015704us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:32.532043 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:32.532239 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.532265 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:32.532307 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:32.532338 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.532394 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.532524 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14587136}
W20260709 12:14:32.546638 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 127114us
I20260709 12:14:32.564162 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:32.564795 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 137239us
I20260709 12:14:32.569679 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:32.574246 23783 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:32.546239 27474 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:32.567885 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 135019us
I20260709 12:14:32.582194 27450 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.594202 23860 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:32.567777 27358 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:32.596092 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 99104.4us
I20260709 12:14:32.566711 27369 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.564524 27224 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:32.601495 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:32.580643 27474 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:32.577028 27443 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:14:32.512776 27480 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:14:32.609077 23783 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:32.619894 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.602077 27346 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.595837 27156 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.610155 27400 raft_consensus.cc:2764] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:14:32.624433 27476 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
W20260709 12:14:32.632784 27486 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:14:32.622211 27427 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 12:14:32.642475 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.644414 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.635839 27369 raft_consensus.cc:2764] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:14:32.663843 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1088) took 1080 ms. Trace:
I20260709 12:14:32.664631 27450 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.665908 27224 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:32.666006 27156 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:32.690860 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.668210 27346 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:32.671096 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.692255 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.658422 27484 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:32.678514 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:32.693627 23851 raft_consensus.cc:2368] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:14:32.650647 27427 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.679775 27438 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.683626 23825 leader_election.cc:336] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.610s (SENT)
I20260709 12:14:32.695192 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.685115 25172 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:32.696413 27225 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.697065 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.697817 27489 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:32.681128 23932 rpcz_store.cc:276] 0709 12:14:31.583062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.610716 (+ 27654us) service_pool.cc:224] Handling call
0709 12:14:32.663630 (+1052914us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b54002a5154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:32.663758 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85217536}
I20260709 12:14:32.704087 23751 leader_election.cc:400] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:14:32.704533 23751 leader_election.cc:403] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:14:32.705400 27484 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.687891 27474 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.688916 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:32.672683 27004 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.694738 27471 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:32.695235 27472 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:32.705904 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 113246us
I20260709 12:14:32.707482 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.709242 27369 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:32.712338 25174 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.716711 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:32.717743 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.720789 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:32.687423 23930 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:32.730690 25171 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:32.699326 27425 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.742857 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:32.743501 25176 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:32.743393 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:32.731563 25171 raft_consensus.cc:606] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:32.732071 27474 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.738150 27369 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:32.746716 25175 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:32.749670 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 169814us
I20260709 12:14:32.738485 27370 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:32.713554 27471 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:32.738653 27225 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.735006 27472 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:32.757771 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:32.765292 27474 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.768018 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:32.751910 27480 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:32.763376 27471 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:32.771560 27425 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:32.772179 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 425) took 1193 ms. Trace:
I20260709 12:14:32.773020 27440 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:32.710511 27438 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.772549 23928 rpcz_store.cc:276] 0709 12:14:31.564351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.564523 (+   172us) service_pool.cc:224] Handling call
0709 12:14:31.564857 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.633786 (+ 68929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.633824 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.633867 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:31.633900 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.757389 (+1123489us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b54000f87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:32.757436 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.757557 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158054784}
W20260709 12:14:32.779213 27483 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 12:14:32.779686 27472 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:32.779235 25176 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:32.772550 25174 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:32.754040 27369 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:32.789343 23898 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:32.790213 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:32.790161 25168 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:14:32.803462 27489 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:32.804131 27489 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:32.805069 27489 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.807061 27369 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:32.788966 27480 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:32.808642 27480 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:32.851848 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2474) took 1200 ms. Trace:
I20260709 12:14:32.852351 23931 rpcz_store.cc:276] 0709 12:14:31.651256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.970062 (+318806us) service_pool.cc:224] Handling call
0709 12:14:32.851028 (+880966us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:32.851246 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:32.851444 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.851470 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:32.851512 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:32.851552 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.851627 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.851755 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16206720}
I20260709 12:14:32.979657 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:32.967331 27004 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:32.996811 27474 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:33.094211 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:33.135375 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 414618us
I20260709 12:14:33.136246 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.101554 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.139941 23860 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:33.140462 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.106227 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.141201 23859 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:33.107877 23855 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:33.109102 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.141990 23854 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:33.104514 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 292255us
I20260709 12:14:33.142800 23855 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:33.145501 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.144622 27370 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:33.149446 23899 leader_election.cc:336] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.531s (SENT)
I20260709 12:14:33.146526 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.179809 27442 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:33.187178 23855 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:33.164521 27224 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.179183 27371 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:33.214974 23859 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:33.217059 23860 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:33.164311 27368 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:33.171018 27358 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.184041 27418 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:33.148567 27474 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:33.243681 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.246152 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:33.247619 23932 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:33.246340 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:33.236148 27484 consensus_meta.cc:287] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:14:33.230425 27368 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.272774 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:33.273526 23933 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:33.273918 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.262243 27474 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.248586 27418 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.244105 27358 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.239115 27471 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.274397 27484 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:14:33.313244 27158 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.248584 27248 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.338761 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.230865 27371 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.274853 27368 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:33.344817 23930 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:33.352138 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.355940 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.360556 23782 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:33.358187 27400 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.362706 27400 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.364795 27400 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:33.369234 23899 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.743s (SENT)
I20260709 12:14:33.369733 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.371580 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:33.372272 23932 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.372687 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.359944 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.374193 23779 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.378508 23933 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:33.375298 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.396894 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.397719 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:33.399149 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.354380 27248 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.369854 27345 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.355647 27507 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.406701 27346 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.396773 27124 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:14:33.411433 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 267438us
I20260709 12:14:33.403995 27439 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.407346 27473 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:33.420976 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.427386 23860 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:33.428506 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.432791 27439 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.433151 23932 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:33.435103 27439 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:33.438591 27418 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.419756 27492 raft_consensus.cc:993] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:33.442320 23779 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:33.409694 27289 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:14:33.427033 27473 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.468778 23782 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:14:33.465835 23851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:33.469866 23851 raft_consensus.cc:606] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.470263 23851 raft_consensus.cc:614] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:33.471853 23900 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.472909 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.474537 27124 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:33.475173 27124 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:33.458468 27476 raft_consensus.cc:1081] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:14:33.477011 27369 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.000s
I20260709 12:14:33.477454 27366 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:33.477741 27501 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:14:33.478104 27366 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:33.479457 27509 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:14:33.476495 27124 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.478497 27471 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.509647 23854 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:33.414880 27507 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.477288 27421 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:33.545711 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 72879.6us
W20260709 12:14:33.555940 23899 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.876s (SENT)
I20260709 12:14:33.556450 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.481419 27473 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:33.558140 27156 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:33.558712 27156 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:33.544131 27442 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:33.500422 27369 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:33.566161 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 420710us
I20260709 12:14:33.567090 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:33.578547 23899 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.902s (SENT)
I20260709 12:14:33.579125 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.589725 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.593048 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.460243 27400 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 12:14:33.562983 27442 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.614513 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.585579 27289 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:33.615916 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.616376 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.619942 23780 raft_consensus.cc:2368] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11.
I20260709 12:14:33.620776 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.621497 23780 raft_consensus.cc:2368] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:33.622294 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.616667 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.617436 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.617825 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.598843 27406 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:14:33.623674 23785 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:33.619128 23776 pending_rounds.cc:85] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:14:33.670876 23776 pending_rounds.cc:107] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:14:33.671164 23776 raft_consensus.cc:2889] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:33.625056 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.625847 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.671547 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.626765 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.672482 23785 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:33.672910 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.623564 23781 raft_consensus.cc:2376] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10. Re-sending same reply.
I20260709 12:14:33.674140 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.629216 27442 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:33.629921 27371 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.631124 27248 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:33.633460 23857 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:33.676657 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.591430 27484 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:33.637570 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.637995 27438 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:33.638790 27366 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:33.640211 27369 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:33.642390 23899 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.048s (SENT)
I20260709 12:14:33.695642 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.697083 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:33.662317 23749 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items [suppressed 13 similar messages]
I20260709 12:14:33.600677 27471 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.665510 27507 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:33.669636 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.677134 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.677456 23785 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:33.569360 27156 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.678351 23854 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:33.613112 27439 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:33.690553 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.695744 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:33.706622 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64272us
I20260709 12:14:33.706746 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.586613 27421 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:33.698554 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.009s	sys 0.038s
I20260709 12:14:33.704698 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.670747 27224 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:33.755117 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.013s	sys 0.045s
I20260709 12:14:33.756024 23779 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:33.711468 23899 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.117s (SENT)
I20260709 12:14:33.774204 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.705708 23783 raft_consensus.cc:2368] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
I20260709 12:14:33.779662 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.780421 23782 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:33.754846 27289 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:33.849061 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 444656us
W20260709 12:14:33.850530 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.755107 27473 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.867645 25172 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:33.861785 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.005s	sys 0.081s
W20260709 12:14:33.870307 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.005s	sys 0.081s
W20260709 12:14:33.870353 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 158925us
W20260709 12:14:33.871024 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 532304us
W20260709 12:14:33.871513 23899 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.174s (SENT)
I20260709 12:14:33.872043 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:33.873750 23899 leader_election.cc:336] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.513s (SENT)
I20260709 12:14:33.874230 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.698685 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2526)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2491)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2530)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2504)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2485)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2508)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2507)
I20260709 12:14:33.881639 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.882366 23783 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:33.886281 27430 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:33.889534 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.890394 23824 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:33.890517 27345 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:33.890882 23824 leader_election.cc:403] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:14:33.875623 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:33.891263 27345 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:33.891820 27480 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.892304 27345 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:33.894688 27371 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:33.900092 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.901865 27371 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:33.890612 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 465) took 1094 ms. Trace:
I20260709 12:14:33.911192 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.912179 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.913355 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.861255 27289 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:33.914101 23932 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:33.910355 23852 rpcz_store.cc:276] 0709 12:14:32.795771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.796144 (+   373us) service_pool.cc:224] Handling call
0709 12:14:32.796445 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:33.132291 (+335846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.132345 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:33.133112 (+   767us) log.cc:844] Serialized 26 byte log entry
0709 12:14:33.439265 (+306153us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:33.439305 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.439371 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.890409 (+451038us) raft_consensus.cc:1685] finished
0709 12:14:33.890432 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.890556 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":294979,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":305500,"threads_started":1,"wal-append.queue_time_us":622051}
I20260709 12:14:33.916384 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.917766 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.796823 27368 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.915314 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.918393 23932 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:33.923556 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:33.927385 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:33.883395 27248 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:33.935062 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.942087 27438 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:33.942781 27438 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:33.947764 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:33.935142 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:33.907991 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:33.960273 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:33.921725 27430 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.923141 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:33.967727 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:33.968403 23925 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:33.963469 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:33.964324 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2508)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
I20260709 12:14:33.978631 23856 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:14:33.955515 27489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.015s	sys 0.021s
I20260709 12:14:33.983428 23749 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2508)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
I20260709 12:14:33.947878 27438 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:34.001485 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:33.995533 23749 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2525)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
I20260709 12:14:33.977342 27346 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:14:34.003662 27438 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:14:34.012115 27438 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:34.012545 27438 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:34.013480 27438 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.023574 27438 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:34.018115 27471 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:34.037065 23778 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:34.042373 27473 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:34.020467 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:34.022596 23783 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:34.007543 27366 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:34.007972 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:34.024979 25119 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:34.004459 23749 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2525)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
I20260709 12:14:34.055254 23784 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.979565 27418 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:34.017791 27522 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.058256 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:34.063784 23855 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.014s	sys 0.015s
W20260709 12:14:34.064610 23855 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.014s	sys 0.015s
W20260709 12:14:33.992494 27489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.016s	sys 0.022s
I20260709 12:14:33.967993 27430 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:34.055028 27366 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:34.130362 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:34.223836 23853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.001s	sys 0.046s
W20260709 12:14:34.350227 23853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.005s	sys 0.057s
W20260709 12:14:34.213521 23934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.014s	sys 0.017s
I20260709 12:14:34.285319 23898 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:34.352254 23934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.016s	sys 0.019s
I20260709 12:14:34.353978 27440 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:34.145169 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.355365 23899 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.691s (SENT)
W20260709 12:14:34.357149 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 71944us
I20260709 12:14:34.061177 27492 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:34.359679 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.376232 23780 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:14:34.079514 27124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.021s	sys 0.026s
I20260709 12:14:34.376972 25114 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:34.378290 25114 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:34.369432 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 403155us
I20260709 12:14:34.354952 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.384177 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:34.383769 23783 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:34.389758 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:34.372391 23899 leader_election.cc:336] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.722s (SENT)
I20260709 12:14:34.391562 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:34.385102 23749 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2512)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2527)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2528)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2531)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
I20260709 12:14:34.242748 27158 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.183964 27421 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:34.121470 27471 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:34.318280 27368 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:34.398437 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 474924us
I20260709 12:14:34.251310 27259 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:34.400378 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:34.356804 27358 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:34.356685 27418 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:34.183815 27400 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:34.408465 23899 leader_election.cc:336] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.732s (SENT)
I20260709 12:14:34.412180 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:34.417655 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.014s	sys 0.004s
I20260709 12:14:34.402717 27442 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:34.418936 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.014s	sys 0.004s
W20260709 12:14:34.440572 23933 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.005s	sys 0.022s
W20260709 12:14:34.448559 23933 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.007s	sys 0.023s
I20260709 12:14:34.405014 27492 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.411075 27158 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.418798 27366 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.437891 27224 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.473400 27438 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:34.771471 27513 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:14:34.418484 27471 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.757997 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:34.727720 23777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.018s	sys 0.014s
I20260709 12:14:34.656765 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.500587 23784 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:34.545886 23899 leader_election.cc:336] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.701s (SENT)
I20260709 12:14:34.763546 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:34.772483 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2837) took 1309 ms. Trace:
I20260709 12:14:34.571638 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2497) took 1254 ms. Trace:
W20260709 12:14:34.783929 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 343990us
I20260709 12:14:34.781049 27438 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.498540 23749 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:43438 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2527)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2533)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2528)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2531)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 587)
I20260709 12:14:34.403263 27519 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:34.417402 27453 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:34.808931 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 574) took 1451 ms (client timeout 1749 ms). Trace:
I20260709 12:14:34.419447 27124 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:34.563097 27543 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:34.814893 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.815311 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2491) took 1194 ms. Trace:
I20260709 12:14:34.815578 23780 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:34.816977 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.815775 23782 rpcz_store.cc:276] 0709 12:14:33.620617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.763223 (+1142606us) service_pool.cc:224] Handling call
0709 12:14:34.815231 (+ 52008us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.420315 27345 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:34.804565 23785 raft_consensus.cc:2410] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:34.810086 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:14:34.805768 23777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.022s	sys 0.014s
I20260709 12:14:34.451233 27489 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.809732 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.808980 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.815488 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:34.759770 27474 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.620971 27368 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:34.809433 23784 rpcz_store.cc:269] 0709 12:14:33.350192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.350308 (+   116us) service_pool.cc:224] Handling call
0709 12:14:34.007847 (+657539us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b5800010598. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68deee 000000000036eb39 00007fb2b27011e0 00007fb2b27ca0c4 00007fb2b27ca131 00007fb2b27780a0 00007fb2b2777984 00007fb2abcf26d4 00007fb2abcf2662
0709 12:14:34.801298 (+793451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3305728}
I20260709 12:14:34.739245 27522 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.806571 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2504) took 1147 ms. Trace:
I20260709 12:14:34.841929 23781 rpcz_store.cc:276] 0709 12:14:33.658836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.656399 (+997563us) service_pool.cc:224] Handling call
0709 12:14:34.806491 (+150092us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.843218 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.782497 27248 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:34.829082 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2526) took 1175 ms. Trace:
I20260709 12:14:34.829387 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2840) took 1362 ms. Trace:
I20260709 12:14:34.780825 27224 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:34.801733 23934 rpcz_store.cc:276] 0709 12:14:33.317345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.317476 (+   131us) service_pool.cc:224] Handling call
0709 12:14:33.317827 (+   351us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:33.349481 (+ 31654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.349516 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:33.356127 (+  6611us) log.cc:844] Serialized 52 byte log entry
0709 12:14:33.704014 (+347887us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b18002d12a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:14:34.353729 (+649715us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:34.353775 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.355739 (+  1964us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:34.526004 (+170265us) raft_consensus.cc:1685] finished
0709 12:14:34.526028 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.571524 (+ 45496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21705,"raft.queue_time_us":21947,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":13285,"spinlock_wait_cycles":29123328,"thread_start_us":649477,"threads_started":1,"wal-append.queue_time_us":1090606}
I20260709 12:14:34.819695 23780 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:34.832397 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.470870 27442 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.838882 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:34.877905 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2510) took 1232 ms. Trace:
I20260709 12:14:34.878393 23778 raft_consensus.cc:2376] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9. Re-sending same reply.
I20260709 12:14:34.878907 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2517) took 1232 ms. Trace:
I20260709 12:14:34.879191 23778 rpcz_store.cc:276] 0709 12:14:33.646898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.814306 (+1167408us) service_pool.cc:224] Handling call
0709 12:14:34.878848 (+ 64542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.472254 27418 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.416978 27259 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.786779 27520 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.804742 27438 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:34.805320 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 259473us
W20260709 12:14:34.889799 23826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.010s
I20260709 12:14:34.845582 23781 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:34.840363 27345 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:34.894013 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.803354 27492 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.877774 27158 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:34.848856 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.420702 27400 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:34.918809 27545 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 12:14:34.919924 27536 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:14:34.920713 27400 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:34.921387 27400 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.921900 27400 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:34.844240 27368 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:34.870502 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:34.829692 23783 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:34.826560 27543 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.924857 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2515) took 1044 ms. Trace:
I20260709 12:14:34.925151 27543 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:34.871654 23779 rpcz_store.cc:276] 0709 12:14:33.653680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.808695 (+1155015us) service_pool.cc:224] Handling call
0709 12:14:34.829017 (+ 20322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.928239 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:34.928839 23779 raft_consensus.cc:2376] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9. Re-sending same reply.
I20260709 12:14:34.929315 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2494) took 1308 ms. Trace:
I20260709 12:14:34.929598 23779 rpcz_store.cc:276] 0709 12:14:33.621391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.927950 (+1306559us) service_pool.cc:224] Handling call
0709 12:14:34.929259 (+  1309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.848937 27522 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.930451 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:34.931013 23779 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:34.872265 23860 rpcz_store.cc:276] 0709 12:14:33.466818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.508884 (+ 42066us) service_pool.cc:224] Handling call
0709 12:14:33.586303 (+ 77419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.787666 (+201363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.787702 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:33.787750 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:33.787782 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.829146 (+1041364us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54004146d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:34.829195 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.829331 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462433152}
I20260709 12:14:34.931838 27522 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:34.831916 27366 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:34.838882 27474 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:34.934181 27474 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:34.828711 27453 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:34.875407 23784 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:34.935011 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2510) took 1055 ms. Trace:
I20260709 12:14:34.935325 23784 rpcz_store.cc:276] 0709 12:14:33.879134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.832121 (+952987us) service_pool.cc:224] Handling call
0709 12:14:34.934946 (+102825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:34.935845 27546 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:34.880864 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.878345 23780 rpcz_store.cc:276] 0709 12:14:33.645193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.816704 (+1171511us) service_pool.cc:224] Handling call
0709 12:14:34.877824 (+ 61120us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:34.890077 23899 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.735s (SENT)
I20260709 12:14:34.801821 23855 rpcz_store.cc:276] 0709 12:14:33.463057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.464204 (+  1147us) service_pool.cc:224] Handling call
0709 12:14:33.464539 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.464786 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:34.350520 (+885734us) log.cc:844] Serialized 14 byte log entry
0709 12:14:34.772123 (+421603us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:34.772174 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:34.772226 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.772297 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.772422 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":848712,"spinlock_wait_cycles":15865216,"thread_start_us":981427,"threads_started":2}
I20260709 12:14:34.891353 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:34.884300 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.021s
W20260709 12:14:34.892925 23826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.007s	sys 0.010s
I20260709 12:14:34.894155 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:34.894923 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:34.906505 23785 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
W20260709 12:14:34.909921 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 495838us
I20260709 12:14:34.935379 27453 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:34.891161 23825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.008s	sys 0.008s
I20260709 12:14:34.875504 27248 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:35.392984 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2527) took 1329 ms (client timeout 1598 ms). Trace:
I20260709 12:14:35.211988 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2481) took 1594 ms. Trace:
I20260709 12:14:34.936391 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:35.128085 23931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.019s
W20260709 12:14:35.394408 23825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.009s	sys 0.009s
I20260709 12:14:35.392736 23779 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:14:35.396716 23931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.004s	sys 0.032s
I20260709 12:14:35.396976 23778 raft_consensus.cc:2368] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8.
W20260709 12:14:35.397100 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.821s (SENT)
W20260709 12:14:35.105836 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 222429us
W20260709 12:14:35.189915 23749 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40134 dropped due to backpressure: service queue is full with 50 items [suppressed 6 similar messages]
W20260709 12:14:35.393360 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 503287us
I20260709 12:14:35.398003 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:35.399011 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.742s (SENT)
W20260709 12:14:35.400074 23899 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.814s (SENT)
W20260709 12:14:35.397804 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.009s	sys 0.039s
W20260709 12:14:35.399734 23782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2495) took 1778 ms (client timeout 2075 ms). Trace:
I20260709 12:14:34.936317 27546 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:35.397100 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2496) took 1775 ms (client timeout 2134 ms). Trace:
I20260709 12:14:35.427063 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:35.020251 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 247215us
I20260709 12:14:35.427866 23780 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10. Re-sending same reply.
I20260709 12:14:35.394833 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2846) took 1428 ms. Trace:
W20260709 12:14:35.400097 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.009s	sys 0.020s
W20260709 12:14:35.428552 23780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2523) took 1780 ms (client timeout 2134 ms). Trace:
I20260709 12:14:35.400594 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:35.402799 23858 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:35.429421 27570 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:35.429514 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.022s
W20260709 12:14:35.432081 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 600744us
I20260709 12:14:35.399181 27224 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:35.429270 23780 rpcz_store.cc:269] 0709 12:14:33.648466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.426737 (+1778271us) service_pool.cc:224] Handling call
0709 12:14:35.428490 (+  1753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.396130 23784 raft_consensus.cc:2410] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:35.434008 23854 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:35.400971 23782 rpcz_store.cc:269] 0709 12:14:33.621630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.873135 (+1251505us) service_pool.cc:224] Handling call
0709 12:14:35.399666 (+526531us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.435175 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2507) took 1790 ms (client timeout 2121 ms). Trace:
I20260709 12:14:35.392688 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2483) took 1774 ms. Trace:
I20260709 12:14:34.925242 23783 rpcz_store.cc:276] 0709 12:14:33.880598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.809479 (+928881us) service_pool.cc:224] Handling call
0709 12:14:34.924793 (+115314us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.437283 23778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2519) took 1789 ms (client timeout 2057 ms). Trace:
W20260709 12:14:35.437819 23778 rpcz_store.cc:269] 0709 12:14:33.647398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.880046 (+1232648us) service_pool.cc:224] Handling call
0709 12:14:35.437210 (+557164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.398095 23781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2522) took 1749 ms (client timeout 2075 ms). Trace:
W20260709 12:14:35.439177 23781 rpcz_store.cc:269] 0709 12:14:33.648202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.893751 (+1245549us) service_pool.cc:224] Handling call
0709 12:14:35.398033 (+504282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:34.945544 27547 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:14:35.179193 27564 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:34.936839 27530 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:14:35.218905 27471 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:35.436559 23776 rpcz_store.cc:276] 0709 12:14:33.618496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.618612 (+   116us) service_pool.cc:224] Handling call
0709 12:14:33.618933 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:33.681626 (+ 62693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.681673 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:33.682203 (+   530us) log.cc:844] Serialized 52 byte log entry
0709 12:14:34.217816 (+535613us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:34.217859 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.218208 (+   349us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:34.921081 (+702873us) raft_consensus.cc:1685] finished
0709 12:14:34.921105 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.392605 (+471500us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8064,"thread_start_us":444763,"threads_started":1,"wal-append.queue_time_us":798444}
W20260709 12:14:35.397575 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:35.435370 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.000s	sys 0.032s
W20260709 12:14:35.399541 27438 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 12:14:35.434685 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:35.427376 27535 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:35.357928 27544 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.940989 27259 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:35.427038 23785 rpcz_store.cc:269] 0709 12:14:34.063348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.848531 (+785183us) service_pool.cc:224] Handling call
0709 12:14:35.392922 (+544391us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.427551 23779 rpcz_store.cc:269] 0709 12:14:33.621885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.930206 (+1308321us) service_pool.cc:224] Handling call
0709 12:14:35.397033 (+466827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.445782 27534 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:34.937532 27492 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:34.943460 27532 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:34.890923 27520 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:35.444921 27440 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:14:35.461515 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.000s	sys 0.032s
W20260709 12:14:35.436375 23784 rpcz_store.cc:269] 0709 12:14:33.644479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.936104 (+1291625us) service_pool.cc:224] Handling call
0709 12:14:35.435111 (+499007us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.466975 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:35.395638 23777 rpcz_store.cc:276] 0709 12:14:33.617095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.617233 (+   138us) service_pool.cc:224] Handling call
0709 12:14:33.617562 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:33.617992 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:14:34.058943 (+440951us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.171764 (+1112821us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:35.172493 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:14:35.172742 (+   249us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:35.173019 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.202197 (+ 29178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.202306 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.211684 (+  9378us) raft_consensus.cc:1685] finished
0709 12:14:35.211709 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.211914 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28851,"prepare.queue_time_us":950166,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":126,"raft.queue_time_us":832666,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":369,"thread_start_us":1403101,"threads_started":3,"wal-append.queue_time_us":850616}
I20260709 12:14:35.482579 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:35.483376 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2533) took 1070 ms. Trace:
I20260709 12:14:35.483711 23777 rpcz_store.cc:276] 0709 12:14:34.414609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.482338 (+1067729us) service_pool.cc:224] Handling call
0709 12:14:35.483307 (+   969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.269845 23750 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40134 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2533)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2528)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2531)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2537)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2538)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2539)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2535)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2542)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2541)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 588)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 589)
I20260709 12:14:35.440084 27519 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:35.439888 27158 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.433748 27546 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:35.474285 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 77217us
W20260709 12:14:35.492916 23825 connection.cc:256] RPC call timeout handler was delayed by -1.1105366829999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 12:14:35.493307 23825 connection.cc:256] RPC call timeout handler was delayed by -1.1088123819999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 12:14:35.493806 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0175026069998694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
W20260709 12:14:35.494078 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0114014549999411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260709 12:14:35.428610 23857 rpcz_store.cc:276] 0709 12:14:33.966345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.967567 (+  1222us) service_pool.cc:224] Handling call
0709 12:14:33.995753 (+ 28186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.473727 (+477974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.473761 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.509411 (+ 35650us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.877071 (+367660us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:34.877113 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.877183 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.244926 (+367743us) raft_consensus.cc:1685] finished
0709 12:14:35.244954 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.394775 (+149821us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11226,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":738974,"threads_started":2,"wal-append.queue_time_us":671635}
I20260709 12:14:35.474735 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:35.474725 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.495373 23856 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:35.495666 23779 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:35.496162 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2529) took 1325 ms. Trace:
I20260709 12:14:35.496481 23779 rpcz_store.cc:276] 0709 12:14:34.170534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.474480 (+1303946us) service_pool.cc:224] Handling call
0709 12:14:35.496098 (+ 21618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.497488 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.498243 23779 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:35.498612 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2531) took 1114 ms. Trace:
I20260709 12:14:35.498903 23779 rpcz_store.cc:276] 0709 12:14:34.384744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.497204 (+1112460us) service_pool.cc:224] Handling call
0709 12:14:35.498555 (+  1351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.457417 27472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.007s	sys 0.033s
I20260709 12:14:35.477959 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:35.504087 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519) took 1622 ms (client timeout 2048 ms). Trace:
I20260709 12:14:35.460664 27418 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:35.469959 27517 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:34.944424 27345 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:35.529806 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 62916.5us
I20260709 12:14:35.533982 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1197) took 1584 ms. Trace:
I20260709 12:14:35.504029 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.475059 27442 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:35.534904 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2532) took 1122 ms. Trace:
I20260709 12:14:35.534384 23852 rpcz_store.cc:276] 0709 12:14:33.949175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.949292 (+   117us) service_pool.cc:224] Handling call
0709 12:14:33.949606 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:33.950005 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.950031 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:33.950495 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.402473 (+451978us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:34.402513 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.402587 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.533769 (+1131182us) raft_consensus.cc:1685] finished
0709 12:14:35.533789 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.533926 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9908,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4929,"thread_start_us":405792,"threads_started":1,"wal-append.queue_time_us":687111}
I20260709 12:14:35.503346 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.535357 23779 rpcz_store.cc:276] 0709 12:14:34.412175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.499564 (+1087389us) service_pool.cc:224] Handling call
0709 12:14:35.534836 (+ 35272us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.504501 23785 rpcz_store.cc:269] 0709 12:14:33.884283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.477700 (+1593417us) service_pool.cc:224] Handling call
0709 12:14:35.504021 (+ 26321us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:35.505657 27366 consensus_meta.cc:287] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.000s
I20260709 12:14:35.537593 23777 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:35.478322 27421 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:35.539870 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:35.540230 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 87309.8us
W20260709 12:14:35.549536 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 149477us
W20260709 12:14:35.552273 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.921s (SENT)
W20260709 12:14:35.553205 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.981s (SENT)
I20260709 12:14:35.541779 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2528) took 1445 ms. Trace:
I20260709 12:14:35.540580 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.553696 23777 rpcz_store.cc:276] 0709 12:14:34.096589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.503037 (+1406448us) service_pool.cc:224] Handling call
0709 12:14:35.541703 (+ 38666us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.477788 27473 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:35.550869 23779 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:14:35.590574 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501) took 1947 ms (client timeout 2503 ms). Trace:
W20260709 12:14:35.590988 23779 rpcz_store.cc:269] 0709 12:14:33.642947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.538831 (+1895884us) service_pool.cc:224] Handling call
0709 12:14:35.590504 (+ 51673us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754880}
I20260709 12:14:35.592073 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:35.541090 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:35.593713 23785 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:35.554489 23784 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:35.532078 27472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.033s
I20260709 12:14:35.554682 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:35.472704 27438 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:35.564167 23899 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.993s (SENT)
I20260709 12:14:35.466028 27564 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:35.473953 27535 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:35.584323 23825 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.921s (SENT)
I20260709 12:14:35.538702 27566 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:35.479485 27534 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:35.614537 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524) took 1961 ms (client timeout 2503 ms). Trace:
W20260709 12:14:35.625640 23785 rpcz_store.cc:269] 0709 12:14:33.653138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.537181 (+1884043us) service_pool.cc:224] Handling call
0709 12:14:35.614459 (+ 77278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.538484 27418 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:35.540297 27224 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:35.617216 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:35.630093 27578 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:35.630667 27578 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:35.551241 27368 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:35.617197 23777 raft_consensus.cc:2368] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:35.593523 27421 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.619076 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520) took 1703 ms (client timeout 2117 ms). Trace:
I20260709 12:14:35.533025 27519 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:35.622989 27569 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:35.616091 27553 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:35.492213 27532 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:35.592887 23779 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:35.573347 27471 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.592151 27473 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:35.538738 27517 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:35.475920 27544 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:35.614802 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 80642.3us
I20260709 12:14:35.536465 27158 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:35.633001 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:35.633682 23777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503) took 1990 ms (client timeout 2610 ms). Trace:
I20260709 12:14:35.551574 27568 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:35.633710 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:35.637238 23784 rpcz_store.cc:269] 0709 12:14:33.915250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.503378 (+1588128us) service_pool.cc:224] Handling call
0709 12:14:35.619011 (+115633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.649804 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2509) took 1719 ms. Trace:
W20260709 12:14:35.619196 23825 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.948s (SENT)
I20260709 12:14:35.624758 27566 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:35.619707 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 133662us
I20260709 12:14:35.709043 27585 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.006011 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.191099 23777 rpcz_store.cc:269] 0709 12:14:33.643475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.554428 (+1910953us) service_pool.cc:224] Handling call
0709 12:14:35.633620 (+ 79192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.192780 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.194103 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:36.197075 23782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516) took 2316 ms (client timeout 2009 ms). Trace:
I20260709 12:14:36.198176 23778 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
W20260709 12:14:36.198329 23780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2529) took 2543 ms (client timeout 2165 ms). Trace:
I20260709 12:14:36.194768 27471 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:36.198720 23780 rpcz_store.cc:269] 0709 12:14:33.654490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.198131 (+2543641us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:36.198262 (+   131us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:36.199692 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.200470 27471 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.195428 23931 rpcz_store.cc:276] 0709 12:14:33.930548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.930699 (+   151us) service_pool.cc:224] Handling call
0709 12:14:33.931003 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.203905 (+272902us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:34.500430 (+296525us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.397177 (+896747us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:35.397653 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:14:35.397782 (+   129us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:35.397815 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:35.431499 (+ 33684us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400334ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:35.431547 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.633154 (+201607us) raft_consensus.cc:1685] finished
0709 12:14:35.649568 (+ 16414us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.649739 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":430601,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":2268,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":36239,"spinlock_wait_cycles":626477312,"thread_start_us":1091748,"threads_started":2,"wal-append.queue_time_us":1091081}
I20260709 12:14:36.201284 27476 raft_consensus.cc:993] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:36.201790 27476 raft_consensus.cc:1081] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:35.708328 27576 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.198624 23778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521) took 2550 ms (client timeout 2203 ms). Trace:
W20260709 12:14:36.204197 23778 rpcz_store.cc:269] 0709 12:14:33.647943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.632707 (+1984764us) service_pool.cc:224] Handling call
0709 12:14:36.198557 (+565850us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.205478 27544 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:36.190366 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 571147us
I20260709 12:14:36.204861 25172 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:35.651008 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.039988 27248 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.189514 27430 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:35.934722 27582 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.206259 23825 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.961s (SENT)
I20260709 12:14:36.190516 27585 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.208834 27358 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:36.205758 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.206820 23785 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9. Re-sending same reply.
I20260709 12:14:36.209462 27358 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.210013 27585 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.212538 27358 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.181421 27124 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:36.208941 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.057s (SENT)
I20260709 12:14:36.203799 27576 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.195643 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1194) took 2261 ms. Trace:
I20260709 12:14:36.200527 23780 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:36.213642 27124 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.214700 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2542) took 1025 ms. Trace:
W20260709 12:14:36.197503 23782 rpcz_store.cc:269] 0709 12:14:33.880858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.196885 (+2316027us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:36.196997 (+   112us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:36.215054 23780 rpcz_store.cc:276] 0709 12:14:35.189366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.199402 (+1010036us) service_pool.cc:224] Handling call
0709 12:14:36.214637 (+ 15235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:35.398154 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40134 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2521)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2520)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2533)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2528)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2531)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2501)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2524)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2503)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2537)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2538)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2539)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2535)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2542)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2541)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 588)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 589)
I20260709 12:14:36.215996 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.214252 23854 rpcz_store.cc:276] 0709 12:14:33.934419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.934530 (+   111us) service_pool.cc:224] Handling call
0709 12:14:33.968278 (+ 33748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.440904 (+1472626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.195328 (+754424us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:36.195532 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370378496}
I20260709 12:14:36.216710 23782 raft_consensus.cc:2376] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9. Re-sending same reply.
I20260709 12:14:36.217242 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2540) took 1067 ms. Trace:
W20260709 12:14:36.217307 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2537) took 1328 ms (client timeout 1691 ms). Trace:
W20260709 12:14:35.777040 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.217619 23782 rpcz_store.cc:276] 0709 12:14:35.149365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.215631 (+1066266us) service_pool.cc:224] Handling call
0709 12:14:36.217161 (+  1530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5504}
I20260709 12:14:36.216020 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.222432 27438 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:35.664637 27421 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.225190 27421 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.225801 27473 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.226615 27582 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.232362 23680 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 7, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:36.220924 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 821) took 1831 ms. Trace:
I20260709 12:14:36.221261 27124 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.222308 23782 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:34.893702 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:36.234865 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2487) took 2615 ms. Trace:
I20260709 12:14:36.235232 23782 rpcz_store.cc:276] 0709 12:14:33.619616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.221805 (+2602189us) service_pool.cc:224] Handling call
0709 12:14:36.222145 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.234575 (+ 12430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.234800 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.236907 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2490) took 2616 ms. Trace:
I20260709 12:14:36.237320 23782 rpcz_store.cc:276] 0709 12:14:33.620384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.236076 (+2615692us) service_pool.cc:224] Handling call
0709 12:14:36.236383 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.236576 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.236601 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:36.236643 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:36.236683 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.236748 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.236853 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:36.212690 27555 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 1.287s	user 0.015s	sys 0.009s
I20260709 12:14:36.237993 27517 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:35.671217 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:36.231074 23781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2522) took 2299 ms (client timeout 1876 ms). Trace:
I20260709 12:14:36.213423 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2538) took 1322 ms. Trace:
I20260709 12:14:36.243759 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:36.244050 23781 rpcz_store.cc:269] 0709 12:14:33.932127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.230886 (+2298759us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:36.231002 (+   116us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:14:36.232275 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.244824 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.245323 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.231235 23780 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:36.239146 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 2844) took 2726 ms. Trace:
I20260709 12:14:36.246666 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.097710 27418 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.246603 23853 rpcz_store.cc:276] 0709 12:14:33.513271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.638683 (+125412us) service_pool.cc:224] Handling call
0709 12:14:33.638973 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.351565 (+712592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.351620 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.360216 (+  8596us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.783456 (+423240us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:34.783498 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.783683 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.212368 (+1428685us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b24003d8b18. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa54b9c4 00007fb2aa549acb 00007fb2aa704dc9 00007fb2aa704a55 00007fb2aa70409d 00007fb2ad33610e 00007fb2ad34d88c 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221
0709 12:14:36.238871 (+ 26503us) raft_consensus.cc:1685] finished
0709 12:14:36.238890 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.239088 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10673,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"spinlock_wait_cycles":67547392,"thread_start_us":1087455,"threads_started":2,"wal-append.queue_time_us":504618}
I20260709 12:14:36.247953 27418 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:36.234051 23680 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 10, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 10 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.248474 27418 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.248546 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2541) took 1087 ms. Trace:
I20260709 12:14:36.249480 23680 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 5, leader changed from <none> to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:36.250622 27418 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.233728 25170 rpcz_store.cc:276] 0709 12:14:34.389353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.396470 (+  7117us) service_pool.cc:224] Handling call
0709 12:14:34.759954 (+363484us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:34.760173 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:34.930641 (+170468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.930676 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:34.930736 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:34.930772 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.220628 (+1289856us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400476654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:36.220684 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.220833 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405713536}
I20260709 12:14:35.635576 27224 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.241077 27582 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.251237 23680 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 8, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.247283 23781 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:36.252929 23680 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.243794 25168 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:36.216357 27576 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.254866 27541 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:36.241792 27532 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.242015 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.210044 23778 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:36.255407 27541 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.256669 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.254884 23680 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.257488 23778 raft_consensus.cc:2410] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:36.258283 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.258829 23778 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:36.245426 23782 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:36.260753 23749 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40134 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 12:14:36.244081 23785 rpcz_store.cc:276] 0709 12:14:34.891030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.626360 (+735330us) service_pool.cc:224] Handling call
0709 12:14:36.213360 (+587000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.260641 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.219884 27400 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.262449 23851 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:36.263258 27534 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.262795 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:36.264212 27546 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.000s
W20260709 12:14:36.264021 23776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2506) took 2619 ms (client timeout 2248 ms). Trace:
W20260709 12:14:36.264724 23776 rpcz_store.cc:269] 0709 12:14:33.644242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.263840 (+2619598us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:36.263954 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:36.249496 23780 rpcz_store.cc:276] 0709 12:14:35.161699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.215769 (+1054070us) service_pool.cc:224] Handling call
0709 12:14:36.248483 (+ 32714us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22016}
I20260709 12:14:36.265750 23778 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
W20260709 12:14:36.177865 23856 consensus_meta.cc:287] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.000s
I20260709 12:14:36.264647 27578 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.266870 23856 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:36.094965 27581 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.268903 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.269791 25176 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:36.269642 27581 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.272060 27581 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.272586 27535 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:36.273473 27554 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.006s	sys 0.013s
W20260709 12:14:36.101614 27573 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.598s	user 0.006s	sys 0.007s
W20260709 12:14:36.275066 27554 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.006s	sys 0.013s
W20260709 12:14:35.907006 27572 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.426s	user 0.000s	sys 0.001s
I20260709 12:14:35.934998 27440 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:35.935575 27571 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.341s	user 0.000s	sys 0.000s
I20260709 12:14:36.277841 27224 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:36.240021 23779 rpcz_store.cc:269] 0709 12:14:34.888581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.591787 (+703206us) service_pool.cc:224] Handling call
0709 12:14:36.217246 (+625459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.197597 23784 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:36.278971 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2535) took 1430 ms. Trace:
I20260709 12:14:36.279350 23784 rpcz_store.cc:276] 0709 12:14:34.848619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.193803 (+1345184us) service_pool.cc:224] Handling call
0709 12:14:36.278905 (+ 85102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.254242 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.280579 23781 raft_consensus.cc:2368] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8.
I20260709 12:14:36.281213 27566 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:35.716259 27567 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:14:36.227941 27473 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:35.839651 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:36.069984 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.004s	sys 0.005s
W20260709 12:14:36.287223 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.005s	sys 0.005s
I20260709 12:14:36.287792 27421 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.288478 27421 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.289073 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:35.955251 27565 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.289776 25174 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:36.289887 27565 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.290936 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.291828 27553 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.292724 27475 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:36.293265 27475 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.294111 27568 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.295188 23777 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:36.296306 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2539) took 1398 ms. Trace:
I20260709 12:14:36.296657 23777 rpcz_store.cc:276] 0709 12:14:34.904915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.192398 (+1287483us) service_pool.cc:224] Handling call
0709 12:14:36.296127 (+103729us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54003a4654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:36.296239 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233599360}
I20260709 12:14:36.299446 27568 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.302451 27541 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.304123 27475 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.305610 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.306706 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1199) took 1425 ms. Trace:
I20260709 12:14:36.307152 23859 rpcz_store.cc:276] 0709 12:14:34.881494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.881617 (+   123us) service_pool.cc:224] Handling call
0709 12:14:34.881905 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:34.882131 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:34.994643 (+112512us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.018883 (+1024240us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b180046ba00. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:36.306154 (+287271us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.306342 (+   188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:36.306388 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:36.306436 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.306507 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.306636 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112145,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":25011,"raft.queue_time_us":153,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":141,"spinlock_wait_cycles":270906752,"thread_start_us":286807,"threads_started":1,"wal-append.queue_time_us":1309769}
W20260709 12:14:36.310958 23783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2518) took 2429 ms (client timeout 1902 ms). Trace:
W20260709 12:14:36.311497 23783 rpcz_store.cc:269] 0709 12:14:33.881394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.310753 (+2429359us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:36.310884 (+   131us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:36.322438 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.323135 27158 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:35.952569 27574 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
W20260709 12:14:35.627756 27543 consensus_meta.cc:287] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.702s	user 0.012s	sys 0.000s
I20260709 12:14:36.293583 27564 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.324795 27543 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.264745 27546 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.327708 27519 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.331301 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.331542 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.332072 23860 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:36.332298 23858 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:36.334131 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.338176 25169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 822) took 1767 ms. Trace:
W20260709 12:14:36.188935 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 625253us
I20260709 12:14:36.268955 27520 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.345513 27421 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.338670 25169 rpcz_store.cc:276] 0709 12:14:34.570600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.570784 (+   184us) service_pool.cc:224] Handling call
0709 12:14:34.599186 (+ 28402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:34.780783 (+181597us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:34.781140 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:14:35.165290 (+384150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:35.165358 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:35.165412 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.336169 (+1170757us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b18003227e0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419
0709 12:14:36.337925 (+  1756us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54007b02d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:36.337961 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.338101 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5976,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":297,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":60435,"spinlock_wait_cycles":679297792,"thread_start_us":383412,"threads_started":1,"wal-append.queue_time_us":774973}
I20260709 12:14:36.354045 27553 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.275931 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:36.355036 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 79189.1us
I20260709 12:14:36.356539 27546 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.358192 23680 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 8, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:36.359982 23680 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 3 to 4, term changed from 3 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.362439 23680 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.364300 23680 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:36.366262 27529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.002s	sys 0.007s
I20260709 12:14:36.365865 23680 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.367671 23680 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.368463 27565 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.374722 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.386324 27553 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:36.387660 23681 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 269) took 18183 ms (client timeout 15000 ms). Trace:
I20260709 12:14:36.387048 27553 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:36.388077 23681 rpcz_store.cc:269] 0709 12:14:18.204395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.204557 (+   162us) service_pool.cc:224] Handling call
0709 12:14:18.356904 (+152347us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5400002490. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68e0b2 00007fb2b3607759 00007fb2b359fa76 00007fb2b3570d58 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc
0709 12:14:36.387519 (+18030615us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:28.963442 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:29.184292 (+220850us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:29.194832 (+ 10540us) write_op.cc:435] Acquired schema lock
0709 12:14:29.195972 (+  1140us) tablet.cc:662] Decoding operations
0709 12:14:29.220690 (+ 24718us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:29.220783 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:29.220812 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:14:29.221838 (+  1026us) tablet.cc:701] Row locks acquired
0709 12:14:29.221886 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:14:29.222068 (+   182us) write_op.cc:270] Start()
0709 12:14:29.222219 (+   151us) write_op.cc:276] Timestamp: P: 1783599269222008 usec, L: 0
0709 12:14:29.222240 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:14:29.274445 (+ 52205us) log.cc:844] Serialized 4369 byte log entry
0709 12:14:31.308142 (+2033697us) op_driver.cc:464] REPLICATION: finished
0709 12:14:31.504427 (+196285us) write_op.cc:301] APPLY: starting
0709 12:14:31.504535 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:31.504780 (+   245us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:31.551169 (+ 46389us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:31.700895 (+149726us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:31.701286 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:14:31.701342 (+    56us) write_op.cc:312] APPLY: finished
0709 12:14:31.701799 (+   457us) log.cc:844] Serialized 83 byte log entry
0709 12:14:31.999793 (+297994us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:32.008709 (+  8916us) write_op.cc:454] Released schema lock
0709 12:14:32.013126 (+  4417us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:32.052219 (+ 39093us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:32.071874 (+ 19655us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20651,"reported_tablets":37,"spinlock_wait_cycles":34728704,"child_traces":[["op",{"apply.queue_time_us":93048,"apply.run_cpu_time_us":29502,"apply.run_wall_time_us":547817,"num_ops":11,"prepare.queue_time_us":106549,"prepare.run_cpu_time_us":16836,"prepare.run_wall_time_us":495111,"raft.queue_time_us":1122298,"raft.run_cpu_time_us":5173,"raft.run_wall_time_us":67172,"replication_time_us":2061783,"thread_start_us":791347,"threads_started":5,"wal-append.queue_time_us":573966,"wal-append.run_cpu_time_us":40277,"wal-append.run_wall_time_us":3071897}]]}
I20260709 12:14:36.394755 27553 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:36.397470 27453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.004s	sys 0.007s
W20260709 12:14:36.397842 27453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.004s	sys 0.007s
W20260709 12:14:36.398281 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.001s	sys 0.009s
W20260709 12:14:36.398526 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.001s	sys 0.009s
I20260709 12:14:36.399319 27594 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.399932 27594 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:36.399529 25177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.004s
W20260709 12:14:36.400914 25177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.001s	sys 0.004s
I20260709 12:14:36.400305 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.402070 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.402171 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.402912 23852 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:36.404076 27158 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:36.406111 27594 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.406037 27158 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.407068 27158 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:36.366783 27529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.002s	sys 0.007s
I20260709 12:14:36.409348 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.409580 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.415381 27158 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.420478 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.421208 23852 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:36.422569 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.429495 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.434974 27158 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:36.438908 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 98339.3us
W20260709 12:14:36.440724 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.150s (SENT)
I20260709 12:14:36.441480 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.441848 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.442193 23779 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:36.442422 23778 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:36.443118 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.465348 23779 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:36.465476 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.466449 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.444715 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.444962 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.467252 23779 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:36.445575 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.468293 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:36.443410 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.009s (SENT)
I20260709 12:14:36.469527 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.470315 23776 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.472159 23857 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:36.261092 23749 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:40134 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2555)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2554)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2550)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2551)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2552)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2549)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2557)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2547)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2556)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 587)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 588)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 589)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 593)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 597)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 598)
Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 599)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2558)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2559)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2560)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2562)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2563)
I20260709 12:14:36.477217 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.478591 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.467216 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.445799 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.446560 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.480603 23780 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:36.446851 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.447543 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.489850 23777 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:36.492127 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.498452 27566 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.501847 23776 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:36.448272 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.513497 23851 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:14:36.449222 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.514642 23782 raft_consensus.cc:2376] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
I20260709 12:14:36.450954 27489 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.515826 27489 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.518082 27489 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.519415 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.521262 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.522111 23856 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:36.523712 27529 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.524304 27529 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.526702 27529 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:36.451812 27600 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:14:36.388514 27543 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:36.454268 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 150294us
I20260709 12:14:36.529623 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:36.530395 27489 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:36.530844 27489 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.460878 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.463331 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.539304 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.539772 23784 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:36.440745 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.443171 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.541404 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:36.464017 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:36.542243 27158 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:36.542265 23852 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:36.542899 27158 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:36.543395 27489 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.543934 27158 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.469604 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.548377 27594 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.549000 27594 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.549860 23779 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:36.512063 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:36.509236 23780 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:36.374168 23680 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:36.535842 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:36.539346 23783 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:36.566582 23680 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.568813 23680 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.551156 27594 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:36.506079 23777 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:36.447929 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:36.573155 27543 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:36.573561 23783 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:36.551539 23854 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:36.575901 27519 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:36.576314 27519 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:36.576716 27519 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.587121 27519 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.589924 27519 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:36.590730 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54982.4us
I20260709 12:14:36.591113 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:36.591814 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.624385 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.624553 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:36.625061 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.625507 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.625824 23781 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:36.625972 27581 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:36.574551 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 126714us
I20260709 12:14:36.626566 27581 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:36.626677 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.627477 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.627962 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.627624 27581 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.628440 23776 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:36.628541 23781 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:36.629313 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.630474 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.631057 23776 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:36.632331 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.632939 23777 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 5.
I20260709 12:14:36.634023 23777 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:36.634588 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2492) took 3013 ms. Trace:
I20260709 12:14:36.634876 23777 rpcz_store.cc:276] 0709 12:14:33.620878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.633588 (+3012710us) service_pool.cc:224] Handling call
0709 12:14:36.633863 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.634368 (+   505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.634533 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.626530 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:36.636611 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.637223 23783 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
I20260709 12:14:36.638427 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.639040 23780 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:36.640200 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:36.640308 27289 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:36.640806 27289 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.641135 27289 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.641382 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:36.642681 27553 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:36.643055 27553 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:36.643424 27553 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.643537 27543 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:36.644037 27543 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:36.644446 27543 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.570218 23680 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 3 to 6, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 6 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:36.650326 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.653333 27553 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.603114 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:36.655798 27158 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:36.656339 27158 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:36.656774 27158 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:36.657343 27553 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.658910 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:36.659682 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52573.4us
I20260709 12:14:36.603130 27569 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:36.623536 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:36.663293 27569 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.664691 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.665480 23932 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:36.669713 23779 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:36.671868 27569 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:36.672636 27158 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.688050 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.689010 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.691206 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.691965 23776 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:36.692906 25170 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:36.605382 23825 connection.cc:256] RPC call timeout handler was delayed by -1.0023498890000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260709 12:14:36.607369 27440 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:36.694656 27440 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.696039 27594 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:36.696837 27594 consensus_queue.cc:1243] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:14:36.696933 27440 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.607942 27587 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:36.697731 27587 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.698601 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2493) took 3077 ms. Trace:
I20260709 12:14:36.698987 23777 rpcz_store.cc:276] 0709 12:14:33.621117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.697751 (+3076634us) service_pool.cc:224] Handling call
0709 12:14:36.698073 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.698270 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.698299 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:36.698341 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:36.698380 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.698434 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.698536 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.699951 27587 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.608070 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.700908 23931 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:36.609371 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.701869 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.609997 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:36.598255 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:36.612052 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.703334 25176 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:36.612668 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.613276 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.704250 23927 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:36.618249 27535 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:36.712448 23780 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.713881 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2502) took 3070 ms. Trace:
I20260709 12:14:36.714516 23928 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.609309 27489 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:36.618906 23899 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.570s (SENT)
I20260709 12:14:36.620432 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.718848 23784 raft_consensus.cc:2376] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
I20260709 12:14:36.622421 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.720144 23782 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:36.629390 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.598824 27566 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:36.626689 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:36.693966 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.700423 23777 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.702646 23932 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:36.610685 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.706718 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.708515 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:36.708920 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.709731 23927 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:14:36.711022 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.712901 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.715467 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2500) took 3076 ms. Trace:
I20260709 12:14:36.715667 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:36.714277 23780 rpcz_store.cc:276] 0709 12:14:33.643221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.712027 (+3068806us) service_pool.cc:224] Handling call
0709 12:14:36.712289 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.713645 (+  1356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.713829 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:36.717495 23929 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:36.717994 23854 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.721783 23781 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:36.650350 27289 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:36.722954 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:36.723834 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:36.893940 23926 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:37.093241 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 294331us
I20260709 12:14:37.093828 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.833441 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.726912 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:36.726872 23778 raft_consensus.cc:2376] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
W20260709 12:14:36.805864 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.094648 23927 pending_rounds.cc:85] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 3
I20260709 12:14:36.729568 23776 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:36.786409 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:36.833844 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:37.094128 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2498) took 3455 ms. Trace:
I20260709 12:14:37.096387 25176 raft_consensus.cc:2410] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:37.097942 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2504) took 3454 ms. Trace:
I20260709 12:14:36.830467 23784 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:37.096781 23927 pending_rounds.cc:107] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:14:37.101914 23776 rpcz_store.cc:276] 0709 12:14:33.643746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.728923 (+3085177us) service_pool.cc:224] Handling call
0709 12:14:36.729235 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.097318 (+368083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.097885 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.094658 23783 rpcz_store.cc:276] 0709 12:14:33.639108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.714749 (+3075641us) service_pool.cc:224] Handling call
0709 12:14:36.715020 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.715178 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.715201 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:36.715241 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:36.715272 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.715321 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.715415 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.100730 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.102923 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2513) took 3457 ms. Trace:
I20260709 12:14:37.105032 23784 rpcz_store.cc:276] 0709 12:14:33.645909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.719535 (+3073626us) service_pool.cc:224] Handling call
0709 12:14:36.830255 (+110720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.102677 (+272422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.102867 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.098335 23777 rpcz_store.cc:276] 0709 12:14:33.638511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.699966 (+3061455us) service_pool.cc:224] Handling call
0709 12:14:36.700259 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.093872 (+393613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.094067 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.106258 23784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f333e0a484248ce88f0905e18500838"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:37.106941 23784 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:37.107381 23784 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.107702 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 583) took 3186 ms. Trace:
I20260709 12:14:37.108103 23777 rpcz_store.cc:276] 0709 12:14:33.921030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.106857 (+3185827us) service_pool.cc:224] Handling call
0709 12:14:37.107146 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.107336 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.107362 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.107403 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:37.107442 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.107500 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.107641 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.102252 23927 raft_consensus.cc:2889] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.097774 23925 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.871050 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.114053 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 584) took 3192 ms. Trace:
I20260709 12:14:37.095381 23930 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:37.110590 23780 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:37.111264 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.110590 23778 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:37.117419 23784 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:36.727567 27587 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.391s
I20260709 12:14:37.118419 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2502) took 3463 ms. Trace:
I20260709 12:14:37.119153 23780 rpcz_store.cc:276] 0709 12:14:33.655021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.110018 (+3454997us) service_pool.cc:224] Handling call
0709 12:14:37.110426 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.116517 (+  6091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.118283 (+  1766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63872}
I20260709 12:14:37.120590 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.120810 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 585) took 3199 ms. Trace:
I20260709 12:14:37.121105 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.121146 23780 rpcz_store.cc:276] 0709 12:14:33.921537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.120027 (+3198490us) service_pool.cc:224] Handling call
0709 12:14:37.120327 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.120497 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.120521 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.120561 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:37.120599 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.120649 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.120761 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.113013 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 581) took 3192 ms. Trace:
W20260709 12:14:37.115299 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.123708 23783 rpcz_store.cc:276] 0709 12:14:33.920530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.112219 (+3191689us) service_pool.cc:224] Handling call
0709 12:14:37.112522 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.112702 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.112727 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.112773 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:37.112802 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.112852 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.112954 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.124390 23777 rpcz_store.cc:276] 0709 12:14:33.921295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.113299 (+3192004us) service_pool.cc:224] Handling call
0709 12:14:37.113579 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.113748 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.113773 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.113811 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:37.113842 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.113891 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.113984 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:37.116858 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 422215us
I20260709 12:14:37.131145 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:36.869451 23785 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:14:37.132094 23785 pending_rounds.cc:85] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 4
I20260709 12:14:37.112643 23776 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:37.132267 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.132781 23785 pending_rounds.cc:107] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:14:37.132894 23926 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:37.133070 23785 raft_consensus.cc:2889] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.126852 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:37.116356 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 393462us
I20260709 12:14:37.129005 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:37.117578 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2527) took 3463 ms. Trace:
I20260709 12:14:37.134797 23783 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:37.135502 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 587) took 2750 ms. Trace:
I20260709 12:14:37.135898 23783 rpcz_store.cc:276] 0709 12:14:34.385396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.134265 (+2748869us) service_pool.cc:224] Handling call
0709 12:14:37.134599 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.135240 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.135440 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.060482 27472 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:37.125591 23784 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:37.138342 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 519420us
I20260709 12:14:37.134409 23778 rpcz_store.cc:276] 0709 12:14:33.653942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.110067 (+3456125us) service_pool.cc:224] Handling call
0709 12:14:37.110412 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.117347 (+  6935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.117519 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8064}
I20260709 12:14:37.140813 23783 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:37.141149 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:37.141641 23925 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:36.924817 27453 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.144203 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2536) took 2264 ms. Trace:
I20260709 12:14:37.144464 23931 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:37.144660 23934 raft_consensus.cc:2368] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:37.144596 23783 rpcz_store.cc:276] 0709 12:14:34.879327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.136743 (+2257416us) service_pool.cc:224] Handling call
0709 12:14:37.140494 (+  3751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.143905 (+  3411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.144144 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.144210 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2514) took 3263 ms. Trace:
I20260709 12:14:37.146062 23776 rpcz_store.cc:276] 0709 12:14:33.880340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.112038 (+3231698us) service_pool.cc:224] Handling call
0709 12:14:37.112335 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.132542 (+ 20207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.143626 (+ 11084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.146413 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 588) took 1997 ms. Trace:
I20260709 12:14:37.124081 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2518) took 3476 ms. Trace:
I20260709 12:14:36.928690 27158 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:37.146850 23778 rpcz_store.cc:276] 0709 12:14:35.148959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.145575 (+1996616us) service_pool.cc:224] Handling call
0709 12:14:37.145903 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.146079 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.146105 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.146150 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:37.146183 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.146233 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.146335 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.155668 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.119096 27553 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:37.150419 23783 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:37.119638 27345 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:37.151117 23825 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.075s (SENT)
I20260709 12:14:36.725734 27535 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:37.156507 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.227s (SENT)
I20260709 12:14:37.157199 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2512) took 3511 ms. Trace:
I20260709 12:14:37.159034 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 582) took 3222 ms. Trace:
I20260709 12:14:37.159514 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 589) took 1982 ms. Trace:
I20260709 12:14:37.148830 23781 rpcz_store.cc:276] 0709 12:14:33.647132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.721093 (+3073961us) service_pool.cc:224] Handling call
0709 12:14:36.721424 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.123494 (+402070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.123970 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:37.170450 23825 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.134s (SENT)
I20260709 12:14:37.175088 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:37.175304 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:37.171185 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.171690 23783 rpcz_store.cc:276] 0709 12:14:35.177520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.145672 (+1968152us) service_pool.cc:224] Handling call
0709 12:14:37.150110 (+  4438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.159049 (+  8939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.159461 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.177623 23783 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:37.178071 23783 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:37.178651 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.179356 23930 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:37.171401 23785 rpcz_store.cc:276] 0709 12:14:33.645666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.727404 (+3081738us) service_pool.cc:224] Handling call
0709 12:14:36.869041 (+141637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.156933 (+287892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.157134 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.180039 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.140991 27248 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:37.142496 27472 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:37.182499 23777 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:37.184743 25177 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:37.190212 23777 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:37.191888 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:37.192314 23785 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:37.194574 23777 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:37.195128 23777 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:37.196069 23777 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.173110 23784 rpcz_store.cc:276] 0709 12:14:33.920799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.106072 (+3185273us) service_pool.cc:224] Handling call
0709 12:14:37.143397 (+ 37325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.172333 23778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:37.197280 23777 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:37.200531 23860 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:37.199581 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.199642 23778 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:37.213934 23778 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:37.188004 23783 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:37.156873 27517 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:14:37.184860 27489 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.227840 23931 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:37.218966 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.220305 23783 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:37.232122 23784 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:14:37.201457 27587 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.237442 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.245123 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.245875 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:37.247749 23777 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:14:37.247414 23931 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:37.181310 27553 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.207031 27440 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:37.157091 27453 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.264909 23785 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:37.267429 23783 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:14:37.268008 23783 pending_rounds.cc:85] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:14:37.268304 23783 pending_rounds.cc:107] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:14:37.268597 23783 raft_consensus.cc:2889] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.268867 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:37.269508 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2558) took 1012 ms. Trace:
I20260709 12:14:37.269969 23783 rpcz_store.cc:276] 0709 12:14:36.257185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.177163 (+919978us) service_pool.cc:224] Handling call
0709 12:14:37.177493 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.269146 (+ 91653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.269436 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:37.271413 23855 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:37.258875 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:37.285791 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:37.290385 23855 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:37.295011 23778 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.297188 23778 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.297755 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 599) took 1094 ms. Trace:
I20260709 12:14:37.298069 23778 rpcz_store.cc:276] 0709 12:14:36.203057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.172206 (+969149us) service_pool.cc:224] Handling call
0709 12:14:37.297645 (+125439us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:37.299652 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 156795us
W20260709 12:14:37.300433 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116161us
I20260709 12:14:37.302613 27617 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:37.303752 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 183879us
I20260709 12:14:37.306820 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.308310 27617 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:37.313827 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:37.314072 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 128419us
I20260709 12:14:37.315048 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.315917 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.316260 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:37.317059 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:37.317606 23926 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.325392 27617 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:37.330636 27540 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:14:37.331377 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.332417 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.333312 27440 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.334625 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.335237 23930 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:37.336551 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.337530 23900 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.338572 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.339787 27440 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.340404 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.341073 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:37.316927 23928 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:37.348416 23784 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:37.357843 23778 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:37.283463 27543 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.003s
I20260709 12:14:37.304425 27158 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:14:37.343446 27615 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:14:37.253163 27587 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:37.379545 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.389453 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:37.389353 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48351us
I20260709 12:14:37.421700 23926 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:14:37.423027 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 130164us
W20260709 12:14:37.429562 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 259118us
W20260709 12:14:37.430286 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.203s (SENT) [suppressed 43 similar messages]
W20260709 12:14:37.431083 23825 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.503s (SENT)
I20260709 12:14:37.368934 27618 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:37.437932 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 281433us
I20260709 12:14:37.248543 27472 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.370151 27619 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:37.371217 27543 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.373426 27158 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:37.431699 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.482491 23927 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:37.369755 27540 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:14:37.422129 27289 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:37.496151 23931 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:37.422784 27624 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:37.450677 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 128647us
I20260709 12:14:37.473261 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 593) took 1933 ms. Trace:
I20260709 12:14:37.478477 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.479547 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:37.503685 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 122791us
I20260709 12:14:37.479938 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.480216 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:37.398554 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 55634.9us
I20260709 12:14:37.443562 27594 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.495227 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:37.506855 23856 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.508128 23776 rpcz_store.cc:276] 0709 12:14:35.539633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.146611 (+1606978us) service_pool.cc:224] Handling call
0709 12:14:37.146843 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.147015 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:37.293146 (+146131us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.472926 (+179780us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.472978 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:37.473027 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.473088 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.473187 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81777,"prepare.queue_time_us":146537,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":183052,"thread_start_us":221676,"threads_started":3,"wal-append.queue_time_us":11600}
W20260709 12:14:37.499516 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.443116 27618 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.466379 27619 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.513684 27535 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:37.523918 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.524948 27543 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.525166 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.465644 27345 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:14:37.519476 27624 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:37.528457 27624 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.531487 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:37.520514 27619 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.532799 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.534579 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.518417 27618 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.535861 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:37.518086 27594 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:37.541901 27594 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:37.542440 27594 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.542906 27594 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.528970 23928 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:37.544960 27553 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.547050 27587 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.546802 27631 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:37.547989 27631 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:37.548344 27631 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:37.549140 27631 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.550153 27453 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:37.550343 27631 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.494805 27158 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.483551 27621 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:37.552292 27540 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:37.505107 27472 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.558156 27248 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:37.558609 27248 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:37.559453 27248 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:37.560544 27248 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.561731 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:37.562206 23899 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.581s (SENT)
I20260709 12:14:37.565222 27158 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.570554 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:37.572749 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.572604 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2499) took 3933 ms. Trace:
W20260709 12:14:37.579903 23899 leader_election.cc:336] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.598s (SENT)
I20260709 12:14:37.580564 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:37.584095 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:37.572593 25113 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:37.587198 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:37.589553 23856 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:37.596210 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:37.598037 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:37.601388 23749 connection.cc:256] RPC call timeout handler was delayed by -3.5574767659998088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
W20260709 12:14:37.603857 23749 connection.cc:256] RPC call timeout handler was delayed by -3.4611767659998804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18380000000000002s for a response
W20260709 12:14:37.606693 23749 connection.cc:256] RPC call timeout handler was delayed by -3.2172767659997135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
W20260709 12:14:37.608011 23749 connection.cc:256] RPC call timeout handler was delayed by -3.0939767659997415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 12:14:37.609136 23749 connection.cc:256] RPC call timeout handler was delayed by -2.8599767659998179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2506s for a response
I20260709 12:14:37.577715 23779 rpcz_store.cc:276] 0709 12:14:33.638844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.719772 (+3080928us) service_pool.cc:224] Handling call
0709 12:14:36.870113 (+150341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:36.898827 (+ 28714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:36.898861 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:37.104197 (+205336us) log.cc:844] Serialized 52 byte log entry
0709 12:14:37.286718 (+182521us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:37.424743 (+138025us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.570084 (+145341us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.572278 (+  2194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.572360 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:37.572402 (+    42us) raft_consensus.cc:1685] finished
0709 12:14:37.572413 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.572546 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7555,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":92821,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":52782,"thread_start_us":463914,"threads_started":3,"wal-append.queue_time_us":330562}
I20260709 12:14:37.599615 27581 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:37.609457 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.588346 27475 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:37.598096 27612 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.348s	user 0.000s	sys 0.003s
I20260709 12:14:37.665964 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:37.666967 25177 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:37.593976 27541 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:37.664306 27648 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:37.672688 27647 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:37.676205 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:37.677706 27594 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:37.602511 27345 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.684672 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:37.689635 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 586) took 3766 ms. Trace:
W20260709 12:14:37.691574 27620 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:14:37.708521 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 230736us
I20260709 12:14:37.709893 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2520) took 4062 ms. Trace:
I20260709 12:14:37.767155 23782 rpcz_store.cc:276] 0709 12:14:33.647693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.753907 (+3106214us) service_pool.cc:224] Handling call
0709 12:14:36.898115 (+144208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:37.257505 (+359390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.257541 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:37.273202 (+ 15661us) log.cc:844] Serialized 26 byte log entry
0709 12:14:37.389496 (+116294us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:37.440424 (+ 50928us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.440886 (+   462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.441039 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:37.709631 (+268592us) raft_consensus.cc:1685] finished
0709 12:14:37.709653 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.709812 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13723,"prepare.queue_time_us":53122,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":316072,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":153,"thread_start_us":451673,"threads_started":2,"wal-append.queue_time_us":392967}
I20260709 12:14:37.708779 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:37.769227 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 60551.3us
W20260709 12:14:37.769937 23825 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.680s (SENT)
W20260709 12:14:37.770911 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.755s (SENT)
W20260709 12:14:37.771827 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.782s (SENT)
I20260709 12:14:37.772245 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:37.733989 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57845us
I20260709 12:14:37.714084 25168 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:37.762431 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 251208us
W20260709 12:14:37.747165 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.704704 27618 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:37.764375 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 79811.4us
I20260709 12:14:37.763161 23780 rpcz_store.cc:276] 0709 12:14:33.922873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.124107 (+3201234us) service_pool.cc:224] Handling call
0709 12:14:37.143164 (+ 19057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.143366 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:37.294571 (+151205us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.689277 (+394706us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.689337 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:37.689386 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.689449 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.689568 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149877,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":106955,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":84,"thread_start_us":519292,"threads_started":3,"wal-append.queue_time_us":199839}
I20260709 12:14:37.775218 23749 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.782997 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.768539 27517 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:37.774677 27540 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:37.794080 23785 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:37.795151 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 186607us
I20260709 12:14:37.794246 23926 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:37.797452 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:37.801185 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.800616 23785 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:14:37.833978 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62186.7us
W20260709 12:14:37.837636 23825 leader_election.cc:336] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.825s (SENT)
I20260709 12:14:37.838176 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:37.838984 27644 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:37.841339 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.973s (SENT)
I20260709 12:14:37.841864 27644 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:37.842263 23825 leader_election.cc:336] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.121s (SENT)
I20260709 12:14:37.842691 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:37.842423 27644 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:37.843845 27659 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:37.844771 23825 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.248s (SENT)
W20260709 12:14:37.845692 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.845839 27644 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:37.847807 27646 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:37.794996 27540 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:37.797303 27517 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:37.793411 27657 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:37.855046 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.856125 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:37.855068 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 351425us
W20260709 12:14:37.858013 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 257886us
I20260709 12:14:37.859453 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.864624 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:37.866869 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:37.876820 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:37.852787 27540 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.879328 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.880249 23854 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:14:37.880612 23854 pending_rounds.cc:85] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 7
I20260709 12:14:37.880837 23854 pending_rounds.cc:107] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:14:37.881072 23854 raft_consensus.cc:2889] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.881594 23860 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:37.886590 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:37.863803 27662 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:37.865850 27649 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:37.900682 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:37.901243 23852 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 12:14:37.914283 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54018us
I20260709 12:14:37.896086 27627 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:37.862723 27581 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:14:37.873695 27660 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:14:37.909307 27623 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:14:37.935704 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.916890 27517 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.162559 23860 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:37.998070 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.165102 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.165992 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:38.166584 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:37.999341 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.168156 23778 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:38.168849 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.001266 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.163132 23860 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:38.170552 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.171674 23782 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:38.003897 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.005551 23785 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:37.865401 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.013015 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.016091 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:37.939343 27345 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:38.176023 27345 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:37.965392 27540 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.177186 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.178699 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:37.997437 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.180070 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.952804 27569 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:37.952937 27618 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:37.963896 27619 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:37.939209 27654 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:14:38.186370 27569 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:37.943073 27661 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:14:38.147946 27663 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:38.159881 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.884s (SENT)
I20260709 12:14:38.169252 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.172108 27651 raft_consensus.cc:2764] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:38.173624 23779 raft_consensus.cc:2368] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:38.179177 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.179574 23781 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:38.180270 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.171070 23860 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:38.182489 27158 consensus_meta.cc:287] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.548s	user 0.016s	sys 0.006s
I20260709 12:14:38.195863 27658 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:38.196514 27540 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:38.213490 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 323726us
I20260709 12:14:38.196848 27345 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.197912 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.200585 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.203011 27619 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:38.196878 23777 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:38.188436 27569 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.216708 23784 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
W20260709 12:14:38.231401 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.231256 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.233529 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:38.234164 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 180387us
I20260709 12:14:38.234503 23777 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:38.234514 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.182812 27660 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:38.240844 27660 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.240044 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.240001 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.244135 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.240753 23778 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:38.245139 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:38.238433 25119 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.625s (SENT)
I20260709 12:14:38.245477 23860 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.248919 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.249204 23780 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7.
I20260709 12:14:38.250303 27566 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:38.251137 27566 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.251964 27658 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.253521 27566 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:38.256155 27658 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:38.274075 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:38.275439 23926 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:38.265619 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:38.289243 27345 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:38.290584 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.292786 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:38.300313 27660 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:38.302199 25177 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:38.302706 25177 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:38.305603 23854 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:14:38.316252 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48041.5us
I20260709 12:14:38.336277 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:38.337473 23852 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:38.342589 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.345016 27543 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:38.343487 23782 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:38.346323 27543 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:38.347978 27543 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:38.350114 27543 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.351902 27543 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:38.352176 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:38.359915 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.362109 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.308382 25177 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.366277 27678 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:38.368096 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.368522 27158 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:14:38.371122 27569 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.371788 27651 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:38.371891 27648 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:38.372591 27651 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.373880 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.374733 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.375281 27651 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:38.375759 27475 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:38.376241 27648 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:38.366360 25177 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:38.389283 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 82601.9us
W20260709 12:14:38.395231 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.401926 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61992.8us
I20260709 12:14:38.404896 23855 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:38.406975 25177 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:38.405645 23682 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 9, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:38.415824 23860 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:38.402886 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.368100 27678 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:38.416780 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:38.412817 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:38.419807 23781 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:38.427963 27566 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:38.429724 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.432329 27623 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:38.433322 23682 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.435395 23682 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:38.444169 27678 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:38.419435 27651 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.451056 23682 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 8, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.439946 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 166) took 7041 ms. Trace:
I20260709 12:14:38.441179 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.451632 27651 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.455843 27678 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:38.437945 27473 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:38.434414 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.456444 27519 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:38.463301 27651 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:38.474570 27659 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:38.476099 27642 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:38.477308 23682 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 9, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:38.497638 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.496912 27623 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:38.457170 23680 rpcz_store.cc:276] 0709 12:14:31.390241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.394745 (+  4504us) service_pool.cc:224] Handling call
0709 12:14:38.431466 (+7036721us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:37.279860 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:37.280440 (+   580us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:37.280480 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:14:37.280500 (+    20us) tablet.cc:662] Decoding operations
0709 12:14:37.442945 (+162445us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:37.484192 (+ 41247us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:37.491427 (+  7235us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:14:37.512977 (+ 21550us) tablet.cc:701] Row locks acquired
0709 12:14:37.512998 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:14:37.513163 (+   165us) write_op.cc:270] Start()
0709 12:14:37.513302 (+   139us) write_op.cc:276] Timestamp: P: 1783599277513099 usec, L: 0
0709 12:14:37.513327 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:14:37.568363 (+ 55036us) log.cc:844] Serialized 6358 byte log entry
0709 12:14:37.939239 (+370876us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7ade28 00007fb2b11e8581 00007fb2b11e7d58 00007fb2b11ec9f1 00007fb2b11ec9a9 00007fb2b11ec939 00007fb2b11ec901 00007fb2b11ebb2d 0000000000388634 0000000000388479
0709 12:14:37.967160 (+ 27921us) op_driver.cc:464] REPLICATION: finished
0709 12:14:37.968609 (+  1449us) write_op.cc:301] APPLY: starting
0709 12:14:37.968708 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:37.969168 (+   460us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:37.969196 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:38.163134 (+193938us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:38.172584 (+  9450us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:14:38.181389 (+  8805us) write_op.cc:312] APPLY: finished
0709 12:14:38.197938 (+ 16549us) log.cc:844] Serialized 113 byte log entry
0709 12:14:38.198280 (+   342us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:38.198879 (+   599us) write_op.cc:454] Released schema lock
0709 12:14:38.198984 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:38.199682 (+   698us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:38.199718 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2708356,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":1160,"apply.run_cpu_time_us":6753,"apply.run_wall_time_us":230938,"num_ops":16,"prepare.queue_time_us":621145,"prepare.run_cpu_time_us":51849,"prepare.run_wall_time_us":659607,"raft.queue_time_us":14257,"raft.run_cpu_time_us":1319,"raft.run_wall_time_us":1342,"replication_time_us":453633,"spinlock_wait_cycles":6347008,"thread_start_us":795209,"threads_started":4,"wal-append.queue_time_us":371274}]]}
W20260709 12:14:38.537770 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 103482us
W20260709 12:14:38.584429 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 82848.7us
W20260709 12:14:38.585000 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.600759 27693 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:38.602870 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 76216.8us
I20260709 12:14:38.624238 23824 leader_election.cc:400] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:38.635159 23824 leader_election.cc:403] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:14:38.618083 23682 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.634423 27694 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:14:38.646116 27683 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:14:38.643122 23682 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 4 to 5, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.605512 27656 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
W20260709 12:14:38.676448 27690 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:14:38.677711 27670 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
W20260709 12:14:38.678603 27687 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:14:38.680500 27686 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.140s	user 0.001s	sys 0.001s
W20260709 12:14:38.682123 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 110506us
W20260709 12:14:38.684284 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 60314.5us
I20260709 12:14:38.690088 27694 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:38.691416 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:38.694075 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:38.696734 27696 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:38.697247 27696 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:38.697646 27696 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:38.708081 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 140346us
I20260709 12:14:38.708627 27656 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:38.709451 27656 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:38.710054 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.711187 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:38.712117 25177 raft_consensus.cc:606] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.712491 25177 raft_consensus.cc:614] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:38.717464 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:38.718190 23926 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:38.695501 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:38.715466 27569 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:38.743985 27701 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:38.744616 27701 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:38.745792 27701 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.752056 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:38.752406 23857 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:38.752888 23860 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.752960 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.752233 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:38.754755 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.755151 27543 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:38.755542 27543 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:38.756158 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.757892 23854 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:38.756175 25168 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:38.764587 23926 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:38.781647 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:38.793730 27543 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.795635 27543 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.793754 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:38.797864 23856 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:38.809966 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:38.813369 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.813804 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.814256 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.815964 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:38.816676 25170 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:38.817075 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:38.822955 25168 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:38.797863 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.825281 23860 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:38.769766 27651 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:38.841670 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46056.4us
I20260709 12:14:38.843811 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24dc0370e122479081184511cb1224f1"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:14:38.844491 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 103326us
I20260709 12:14:38.844653 25170 raft_consensus.cc:606] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:38.845474 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:38.847023 27440 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:38.847451 27440 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:38.847803 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:38.848770 27623 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:38.844296 27519 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:38.849187 27623 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:38.853713 27623 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.855533 23925 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:38.856532 27651 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.849203 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:38.853881 27519 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:38.865911 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:38.866807 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "643746a7a11243df94746cf1081fc078"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:14:38.866936 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 141411us
I20260709 12:14:38.847890 27440 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:38.868973 25177 raft_consensus.cc:606] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:38.859510 27519 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.859019 27651 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:38.875368 25177 raft_consensus.cc:614] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:38.876700 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:38.877996 27566 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:38.878479 27566 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:38.878896 27566 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.879074 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:38.881807 27535 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:38.882468 27535 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.882957 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:38.888803 27623 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.889716 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:38.885907 27569 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.892108 27569 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.847693 27696 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.920100 23852 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:38.921291 27696 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:38.893440 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:38.895386 27517 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:38.895763 27248 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:38.931748 27623 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:38.935016 23852 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:38.935889 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.896632 27569 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:38.897289 27535 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:38.899070 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:38.936978 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.895442 27587 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:14:38.901054 27651 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:38.901222 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:38.937625 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.909107 27659 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:38.910481 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:38.938562 27705 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:38.938719 27651 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:38.917259 27624 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:38.938254 27587 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:38.939829 27587 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:38.940574 27624 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:38.940973 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:38.940810 27587 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.942447 27587 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.942574 27624 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:38.914455 25177 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:38.918515 27581 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:38.938866 27659 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:38.939352 27651 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:38.946242 23931 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:38.946964 27651 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.939852 27705 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:38.917793 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:38.945945 27659 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.944272 27656 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.950132 27659 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.949929 27535 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:38.948421 27651 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.956468 27517 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:38.963930 23926 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:38.943251 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:38.959503 27656 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:38.983759 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:38.988963 27519 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:38.990262 27701 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:38.990707 27701 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:38.991197 27701 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:38.991743 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:39.000816 23858 raft_consensus.cc:606] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:39.001189 23858 raft_consensus.cc:614] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:38.935436 27440 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:38.996824 27617 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.996533 27540 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.004664 27617 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.005563 27540 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.006178 27440 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.007136 27617 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:39.009337 27696 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:39.010329 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:39.011152 23852 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:39.009564 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:39.012207 27566 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.014410 27566 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:38.991557 27519 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:39.015688 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:39.016379 25177 raft_consensus.cc:606] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:39.018421 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:39.019286 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:39.019564 27663 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.020005 25169 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:39.020129 27663 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.021198 27663 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.023864 27540 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:39.028693 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71572.2us
I20260709 12:14:39.033838 23926 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:39.033747 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:39.041783 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.045749 23931 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.039136 27651 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:14:39.049213 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:39.050695 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:39.052624 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.054750 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:39.055564 25170 raft_consensus.cc:606] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:39.055868 27519 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.056298 27519 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.056651 27519 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.047556 27696 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.045732 27656 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:14:39.060300 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:39.062434 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:39.067174 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:39.068154 23782 raft_consensus.cc:2410] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:14:39.073446 23899 leader_election.cc:336] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.045s (SENT)
I20260709 12:14:39.074045 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:39.075671 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46474.8us
I20260709 12:14:39.086987 23931 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:39.097836 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.099103 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:39.100430 27696 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.100893 27696 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:39.101280 27696 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.102129 23852 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:39.102388 23858 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:39.108608 23860 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:39.109977 27642 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:14:39.103678 27248 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.108608 27705 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:39.123698 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:39.124006 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60157.8us
I20260709 12:14:39.125111 27248 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.126564 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.133841 23926 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:39.134855 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:39.135682 23933 raft_consensus.cc:2393] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:39.125733 27248 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.137558 27248 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.138532 23925 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:39.139915 23925 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:39.141307 23926 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:39.152380 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.153265 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.158279 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:39.159952 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:39.160436 27721 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.160789 23930 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:39.161644 27693 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:39.162082 27693 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:39.163004 27693 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.164311 27693 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:39.165083 23899 leader_election.cc:336] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.135s (SENT)
I20260709 12:14:39.165563 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:39.168083 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 120806us
I20260709 12:14:39.169095 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:39.170845 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.131870 27705 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:39.175030 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 123342us
I20260709 12:14:39.174818 27705 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.173664 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:39.179867 25169 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:39.189815 27724 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:39.190013 23899 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.150s (SENT)
W20260709 12:14:39.191293 23899 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.231s (SENT)
I20260709 12:14:39.192080 27440 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:39.210672 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58509.7us
I20260709 12:14:39.192389 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:39.191994 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:39.195314 27709 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.198717 27517 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:39.187634 27712 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.095s	user 0.001s	sys 0.001s
I20260709 12:14:39.199311 27617 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:14:39.202284 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.250919 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:39.254208 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.504s (SENT)
I20260709 12:14:39.211095 27663 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:39.250284 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.266860 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.284037 27248 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.284550 27248 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.284942 27248 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.294103 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.294183 27158 raft_consensus.cc:993] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.295183 27158 raft_consensus.cc:1081] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.231427 27540 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:14:39.295212 27718 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:39.296530 27718 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.259125 27663 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:39.297389 27718 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.297878 27543 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.298491 27718 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.283346 27519 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.300756 27631 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:14:39.301805 27631 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:39.302181 27631 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:39.303265 27617 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:39.303723 27617 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:39.304617 27617 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.272917 27541 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.316766 27721 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:39.318006 27721 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:39.318379 27721 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.319820 27663 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:39.321322 23932 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:39.322685 27617 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:39.323783 27699 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.326220 27709 raft_consensus.cc:993] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.326905 27700 raft_consensus.cc:1081] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.328042 27345 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:39.328510 27345 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:39.329612 27345 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.236531 27724 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:14:39.332882 27724 consensus_queue.cc:1243] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:39.333490 27724 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:39.333840 27724 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:39.337175 27720 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:14:39.319357 27721 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.349340 27721 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.334889 27724 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.365887 27724 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.303143 27631 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:39.384153 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.482376 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 234389us
W20260709 12:14:39.471851 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 280595us
W20260709 12:14:39.489704 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 197103us
W20260709 12:14:39.490677 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 238253us
I20260709 12:14:39.491900 23782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24dc0370e122479081184511cb1224f1"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:39.492821 27543 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:39.494845 23776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:39.497749 23782 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:39.498003 23776 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:39.498572 23776 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.498178 23782 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:39.499970 23931 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:39.501267 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:39.496266 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 325532us
I20260709 12:14:39.499106 27519 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:39.517175 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.555s (SENT)
I20260709 12:14:39.500339 27631 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:39.517119 27475 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.539768 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.540657 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:39.512871 27345 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.556555 27696 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.558992 27696 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:39.559839 23858 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:39.565285 27659 consensus_meta.cc:287] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.615s	user 0.014s	sys 0.004s
I20260709 12:14:39.566097 23856 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:39.571751 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.574594 27701 consensus_meta.cc:287] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.005s
I20260709 12:14:39.575059 27701 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.576277 23859 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:39.577348 27701 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.579964 27730 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:39.581220 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49887.3us
I20260709 12:14:39.582147 27248 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.582587 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.585704 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:39.579250 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:39.590126 27730 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:39.595623 27730 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.596293 27730 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:39.598349 27541 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.609804 27663 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:39.624447 27734 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:14:39.674052 27663 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.616626 27648 raft_consensus.cc:993] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.675350 27648 raft_consensus.cc:1081] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:39.604018 27543 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.663700 27724 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:14:39.678542 27724 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:39.678934 27724 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.680298 27724 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.599547 27730 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:39.681537 27724 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:39.646536 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:39.643193 25177 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:39.624316 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:39.684177 23859 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:39.616890 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64677.1us
I20260709 12:14:39.687772 27248 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.615258 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:39.689388 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 74261.5us
I20260709 12:14:39.689497 27730 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:39.691435 27718 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:39.567473 27699 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.697505 25169 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:39.703145 23825 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.893s (SENT)
I20260709 12:14:39.705761 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:39.706385 23931 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.670603 27475 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:39.717399 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:39.718025 23780 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:39.718366 23780 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.729792 23931 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:39.666947 23782 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:39.730886 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 94060.9us
I20260709 12:14:39.722869 25168 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:39.753420 25169 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:39.738189 23780 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.660447 23852 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:39.690696 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:39.764075 23782 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.663407 23776 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.766040 23777 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:39.767159 23781 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:39.768096 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 63695.5us
I20260709 12:14:39.771648 23780 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:39.773939 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83388.6us
W20260709 12:14:39.775457 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.904s (SENT)
I20260709 12:14:39.771001 27663 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.780357 23851 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:39.776611 27663 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:39.781265 27663 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.637131 27722 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.137s
I20260709 12:14:39.780652 23776 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.788591 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:39.795336 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.794061 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:39.799001 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:39.799702 23856 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.801673 23859 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:39.805058 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.806247 27721 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:39.807451 27721 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:39.807722 27718 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:39.808588 27718 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:39.808900 27716 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.808588 27705 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:39.809479 27716 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:39.802966 23900 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:39.811902 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:39.814616 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:39.814589 23858 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:39.818694 27517 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.819796 27716 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.820849 27663 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.823135 27663 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:39.811158 27721 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.819100 27729 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:14:39.825593 27729 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:39.825955 27729 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:39.826638 27735 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:14:39.828212 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 44875.6us
I20260709 12:14:39.829151 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.830060 27730 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:39.830624 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:39.808926 27718 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.831553 27730 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:39.832134 27730 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.832537 27718 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.833047 27730 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.833849 27718 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.826843 27729 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.780471 27345 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:14:39.803953 27693 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:39.807400 27587 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.842830 23856 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:39.845587 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:39.849553 27693 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:39.850245 27663 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:39.850982 27663 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:39.850608 27693 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.851379 23927 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:39.841341 27517 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:39.855757 27729 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.871575 23680 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:39.875382 23825 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.924s (SENT)
I20260709 12:14:39.876330 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.883765 27701 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:39.872267 27656 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:14:39.892863 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:39.871888 27587 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:39.901628 23858 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:14:39.902134 23680 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:39.903043 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:39.903797 23680 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 7, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:39.905272 23680 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:39.906771 23680 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 13, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 13 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:39.909179 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51371.6us
W20260709 12:14:39.927990 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.932483 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74895.9us
I20260709 12:14:39.921420 27651 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:39.936100 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 64516.9us
I20260709 12:14:39.920500 27701 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.958827 27642 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:39.963559 27642 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.966465 27642 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:39.959144 27701 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.968514 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:39.969343 23930 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:39.969610 27701 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:39.975358 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:39.936504 27651 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:40.015615 23680 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:40.051692 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.203s (SENT)
I20260709 12:14:40.013468 27699 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:39.920975 27663 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:39.974081 27722 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:40.072273 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 169303us
I20260709 12:14:40.061972 23782 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:40.054427 27730 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:40.076860 27722 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.051057 27651 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:40.081852 27722 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.053123 27716 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.084036 27716 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.086197 27716 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:40.061755 27699 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:40.089514 27699 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.090955 27699 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:40.096465 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.935642 27517 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:39.980928 27587 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.116294 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:40.121310 25168 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:40.079917 27730 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:40.116127 27749 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:40.140057 23898 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:40.175004 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 137426us
I20260709 12:14:40.183334 23680 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:40.184980 23680 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:40.186580 23680 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 8, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:14:40.190552 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 50585.6us
I20260709 12:14:40.154857 27721 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.206866 27721 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:40.207394 23825 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.694s (SENT)
I20260709 12:14:40.207855 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.208221 27716 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:40.208652 27716 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.209000 27716 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.202791 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:40.212844 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.207137 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.235085 23780 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:40.238413 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.239145 23856 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:40.240198 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.234900 23932 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
W20260709 12:14:40.242125 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.705s (SENT)
I20260709 12:14:40.242585 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:40.245667 23930 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:40.253510 27742 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:14:40.253113 27730 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.256584 27656 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.245126 27659 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:40.259873 27659 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:40.260317 27659 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.245424 27642 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:40.266461 27747 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
I20260709 12:14:40.210944 27475 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:14:40.274545 27475 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:40.274997 27475 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:40.275916 27475 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.247483 27728 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.278928 27728 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:40.288486 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 313192us
W20260709 12:14:40.296207 23682 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 273) took 20912 ms (client timeout 15000 ms). Trace:
W20260709 12:14:40.298333 23682 rpcz_store.cc:269] 0709 12:14:19.384010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.416913 (+ 32903us) service_pool.cc:224] Handling call
0709 12:14:40.296064 (+20879151us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:39.054407 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:39.055216 (+   809us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:39.055292 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:14:39.055314 (+    22us) tablet.cc:662] Decoding operations
0709 12:14:39.056749 (+  1435us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:39.056854 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:39.056893 (+    39us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:14:39.057828 (+   935us) tablet.cc:701] Row locks acquired
0709 12:14:39.057878 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:14:39.058052 (+   174us) write_op.cc:270] Start()
0709 12:14:39.058202 (+   150us) write_op.cc:276] Timestamp: P: 1783599279057991 usec, L: 0
0709 12:14:39.058225 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:14:39.078499 (+ 20274us) log.cc:844] Serialized 2840 byte log entry
0709 12:14:39.087637 (+  9138us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7ade28 00007fb2b11e8581 00007fb2b11e7d58 00007fb2b11ec9f1 00007fb2b11ec9a9 00007fb2b11ec939 00007fb2b11ec901 00007fb2b11ebb2d 0000000000388634 0000000000388479
0709 12:14:39.293907 (+206270us) op_driver.cc:464] REPLICATION: finished
0709 12:14:39.338689 (+ 44782us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2aa6a80ac 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2b11e8ef0
0709 12:14:39.739421 (+400732us) write_op.cc:301] APPLY: starting
0709 12:14:39.739525 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:39.739801 (+   276us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:39.739853 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:39.743206 (+  3353us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:39.762183 (+ 18977us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:14:39.768588 (+  6405us) write_op.cc:312] APPLY: finished
0709 12:14:39.796466 (+ 27878us) log.cc:844] Serialized 59 byte log entry
0709 12:14:39.811197 (+ 14731us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:39.818473 (+  7276us) write_op.cc:454] Released schema lock
0709 12:14:39.838944 (+ 20471us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:39.856242 (+ 17298us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:39.856276 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6277,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":445124,"apply.run_cpu_time_us":3670,"apply.run_wall_time_us":108780,"num_ops":7,"prepare.queue_time_us":347551,"prepare.run_cpu_time_us":7450,"prepare.run_wall_time_us":36041,"replication_time_us":235483,"spinlock_wait_cycles":103108736,"thread_start_us":505127,"threads_started":2}]]}
I20260709 12:14:40.307860 27728 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:40.310350 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 57351.6us
W20260709 12:14:40.328444 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 86337us
W20260709 12:14:40.329200 23825 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.864s (SENT)
I20260709 12:14:40.329610 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:40.318008 27709 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:14:40.321499 27651 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:40.343015 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 56260.3us
I20260709 12:14:40.340958 27718 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.356426 23776 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:40.343387 27642 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.348193 27475 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.362568 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.364822 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.364903 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.372014 27651 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:40.376075 23899 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.011s (SENT)
I20260709 12:14:40.377094 27756 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:40.379789 27756 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.380041 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.381394 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:40.382694 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:40.365348 27718 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.383719 27663 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.384481 27663 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:40.385200 27663 raft_consensus.cc:2764] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:40.385710 27718 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:40.386059 27663 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:40.386533 27663 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:40.386931 27663 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.367921 27722 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:40.387928 27722 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.388314 27722 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:40.393515 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:40.380159 27756 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:40.396654 23825 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.814s (SENT)
I20260709 12:14:40.398208 23856 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.408550 27748 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:40.414027 23825 leader_election.cc:336] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.165s (SENT)
I20260709 12:14:40.414520 23825 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.414875 27722 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.415315 27543 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:40.424459 23856 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:40.417512 27722 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.425122 27721 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.427325 27721 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.428865 27345 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:40.429240 27345 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:40.429620 27721 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:40.430110 27345 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.431285 27345 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:40.426699 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:40.444089 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74571.2us
W20260709 12:14:40.444888 23899 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.034s (SENT)
I20260709 12:14:40.449617 25168 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:40.451279 27744 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:40.465130 27648 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.465831 27648 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.473256 27648 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:40.475090 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.475932 23855 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:40.479796 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:40.480825 27709 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:40.481256 27709 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.481578 27709 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:40.479522 27659 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.485795 27659 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.486826 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.496325 27721 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.498008 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.499009 27721 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:40.500484 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.501544 27721 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:40.503069 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.503880 27721 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:40.504398 27721 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:40.505266 27721 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.509913 27716 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.511945 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.512980 27718 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.514729 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.517098 27718 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.515879 27709 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.521358 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.522140 23776 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:40.524274 27718 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.524865 27718 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.526311 27744 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:40.523849 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.527289 27718 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.531965 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:40.532693 23852 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:40.545403 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.546844 23852 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:40.552552 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:40.555341 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.556828 23776 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:40.573372 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 207251us
I20260709 12:14:40.575635 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:40.576311 23852 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.554256 27709 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:40.586086 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.587775 23780 raft_consensus.cc:2376] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
I20260709 12:14:40.590063 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.598841 23852 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.571899 27648 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:40.541666 27663 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.602281 23782 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:14:40.603048 23934 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:40.603339 27716 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:40.607923 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:40.609967 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:40.611274 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.611734 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.613265 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.614353 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.622903 23932 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:40.619558 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.624143 23854 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:40.630802 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.610749 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:40.625429 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.633095 23778 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.625485 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.634307 23859 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:40.622830 25169 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.624781 23784 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.649351 27663 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.650660 27345 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.651371 27345 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.622551 27760 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:40.652710 27760 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.654973 27760 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:40.600237 27617 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:40.656055 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.669093 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:40.670825 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44736.5us
I20260709 12:14:40.674046 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:40.671021 23751 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:40.677225 23851 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:40.680804 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.684711 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.685422 23782 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:40.686715 23931 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:40.681480 23934 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:40.637945 27517 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:40.689412 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.692628 23859 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:40.690057 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.625572 27758 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.631095 27648 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:40.683286 25169 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:40.691182 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.666538 27663 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.666214 27345 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.711650 27663 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.698807 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.712913 23783 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:40.699474 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.713877 23778 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.704113 27517 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:40.714838 23825 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.851s (SENT)
I20260709 12:14:40.700927 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.705044 27543 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:40.706621 27473 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:40.679569 27721 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:40.715391 23825 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.715643 23776 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:40.715756 27721 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.704691 27642 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:40.713914 27663 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.716259 27721 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.718673 27648 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.688850 27718 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:40.720088 27718 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.720463 27718 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.717891 23784 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:40.731115 23932 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
W20260709 12:14:40.733538 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.742970 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.746840 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.747835 23855 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
W20260709 12:14:40.758167 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.765700 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:40.766634 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:40.715273 27517 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.768980 27757 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:40.769440 27757 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:40.769795 27757 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:40.766695 27770 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:40.774448 27770 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.774828 27770 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.777776 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.778476 23778 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:40.780040 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.780364 23781 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:40.781658 23783 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:40.782804 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.747548 23780 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:40.782078 23781 pending_rounds.cc:85] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 7
I20260709 12:14:40.792026 23776 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:40.792778 23777 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:40.792992 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.792476 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.795217 23782 raft_consensus.cc:2376] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11. Re-sending same reply.
I20260709 12:14:40.796165 23927 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
W20260709 12:14:40.797097 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54208.7us
I20260709 12:14:40.800658 23784 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:14:40.801205 23784 pending_rounds.cc:85] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 3
I20260709 12:14:40.797679 27656 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:14:40.801893 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57959us
W20260709 12:14:40.803906 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57360.2us
I20260709 12:14:40.715267 27756 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.804653 23855 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:40.794557 23776 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:40.806301 23856 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:40.806301 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:40.801573 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.811309 23777 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.766553 27709 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:40.792522 23781 pending_rounds.cc:107] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:14:40.814699 23781 raft_consensus.cc:2889] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:40.806978 23784 pending_rounds.cc:107] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:14:40.817518 23784 raft_consensus.cc:2889] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:40.818535 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 103739us
I20260709 12:14:40.829005 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.829926 23776 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:40.832675 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.872s (SENT)
I20260709 12:14:40.833199 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:40.834678 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.835436 23776 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.836403 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.836941 23776 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:40.830978 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.837762 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.838361 23776 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:40.839216 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.840415 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:40.842447 23782 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.727530 27721 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:40.843570 23776 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:40.793766 27693 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:40.814395 27709 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:40.728849 27758 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:40.852633 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.852748 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.853549 23784 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:40.801887 27656 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:40.814468 27756 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.872016 23855 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:40.876101 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.876787 23776 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:40.877638 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.855254 27543 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:40.888126 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:40.897581 23930 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.801908 27749 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
W20260709 12:14:40.895844 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 63203.9us
W20260709 12:14:40.898914 23825 leader_election.cc:336] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.900s (SENT)
I20260709 12:14:40.899348 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:40.901116 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.901945 23927 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:40.903492 23782 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:40.909425 25113 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:40.880672 27701 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.911221 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.920565 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:40.922108 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.883343 27758 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:40.922838 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:40.927023 23858 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:40.929239 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:40.924134 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:40.930338 23929 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:40.931911 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 124078us
I20260709 12:14:41.021324 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:40.917007 27648 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:40.901911 27718 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.022588 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:40.930610 27765 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:14:40.884616 27663 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:14:41.022106 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49126.2us
W20260709 12:14:41.040364 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 141463us
W20260709 12:14:41.072993 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.938s (SENT)
I20260709 12:14:41.040647 23776 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:41.073496 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.083927 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.021533 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:41.086594 25177 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:14:41.037098 27648 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:41.072136 23749 connection.cc:256] RPC call timeout handler was delayed by -1.2905830499998956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:14:41.097239 23749 connection.cc:256] RPC call timeout handler was delayed by -1.1267830499998581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
I20260709 12:14:41.095083 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:41.021211 27763 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:41.109503 23825 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.960s (SENT)
I20260709 12:14:41.111838 23926 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:41.113163 27784 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.115877 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:41.088209 27659 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.919251 27709 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.100456 27648 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.141973 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.142724 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:41.145684 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.146314 27777 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:41.150115 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.151070 25171 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:41.151459 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.152153 23782 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:41.156541 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.158077 23776 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:41.125681 27775 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:14:41.126188 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:41.128552 27778 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:14:41.022347 27656 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.131052 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:41.177706 23929 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:41.136252 27663 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.137321 27721 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.178331 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.180691 25177 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:41.183823 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.138105 27770 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.138605 27718 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:41.123893 27784 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.150970 23925 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 6: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:41.155650 27757 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.200649 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.201929 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.170607 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.176538 23930 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:41.203846 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.204936 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.205564 23776 raft_consensus.cc:2368] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:41.206329 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.207211 27648 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.208879 25169 raft_consensus.cc:2368] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7.
I20260709 12:14:41.210706 27648 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.213573 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.214100 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.207304 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.217137 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.217813 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.228910 23930 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:41.216187 23926 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:41.202641 27543 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:41.249145 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.252971 23930 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:41.254019 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.200089 27699 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.270568 23777 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:41.261929 23930 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:41.263478 23929 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:41.273451 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.274233 23783 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:41.275313 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.274416 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.276352 23776 raft_consensus.cc:2368] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:41.182286 27663 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.278676 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.281021 23777 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:41.215936 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:41.273789 23925 raft_consensus.cc:1217] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[8.4-8.4]   Dedup: 8.4->[]
I20260709 12:14:41.283561 23925 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:14:41.285395 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.286270 23777 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:41.294591 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.298930 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.299836 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.300534 23777 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.307543 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:41.277890 27699 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.309129 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:41.309612 27762 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.606s	user 0.005s	sys 0.000s
I20260709 12:14:41.309892 25173 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:41.311278 27699 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.317670 25174 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:41.322853 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.323446 23781 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:41.324306 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.325193 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.326213 25174 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:41.327392 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.328043 23783 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:41.333953 23777 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:14:41.334836 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.335363 23777 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
W20260709 12:14:41.335817 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 215730us
W20260709 12:14:41.337277 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.101s (SENT)
I20260709 12:14:41.337635 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:41.338255 25168 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.341123 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.342213 23777 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:41.343222 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.344620 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:41.354254 23777 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:41.355525 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.356312 23777 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:41.357338 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.358160 23777 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:14:41.363953 27543 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.368880 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.376040 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.385265 27770 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.388679 23776 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: [5.4-9.5]
I20260709 12:14:41.390228 25176 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:41.391316 25175 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:41.394068 25168 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:41.397380 23776 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:14:41.397709 23776 pending_rounds.cc:85] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:14:41.399775 23776 pending_rounds.cc:107] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:14:41.400084 23776 raft_consensus.cc:2889] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:41.403661 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.105s (SENT)
W20260709 12:14:41.409667 23825 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.201s (SENT)
I20260709 12:14:41.410037 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.402261 27757 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:41.407637 27693 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 12:14:41.433530 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.434250 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:41.435141 23899 leader_election.cc:336] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.944s (SENT)
I20260709 12:14:41.408237 27543 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:14:41.439024 23825 leader_election.cc:400] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9.
I20260709 12:14:41.438913 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.443071 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.445804 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.448916 27730 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:41.442901 27791 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:41.443821 27721 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:41.453399 27784 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.438670 27517 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:14:41.456991 23781 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:41.457947 23777 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:41.458534 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.460139 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.461961 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.462597 23931 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.464874 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.467530 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.470528 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:41.458123 27744 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:41.471877 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.472543 23925 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.460431 27721 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:41.473366 27794 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:41.475539 27790 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:41.475862 23778 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:41.478920 23776 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:41.479667 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:41.483901 23927 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:41.485968 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:41.489006 27776 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
I20260709 12:14:41.491955 23927 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:41.492847 27517 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:41.494385 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:41.471231 23929 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 6.
I20260709 12:14:41.509148 23925 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:41.507472 23931 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:41.509542 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:41.517338 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49888.1us
I20260709 12:14:41.518219 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:41.522755 23852 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:41.515863 27744 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:14:41.526010 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:41.526939 23852 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:41.527251 27624 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:41.527755 27624 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:41.528304 27624 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:41.529407 27794 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:41.532442 27631 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:14:41.543804 27746 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:41.544544 23899 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.545352 27746 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:41.545885 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.546653 27746 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:41.547900 27581 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:41.548199 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:41.552009 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:41.521920 27663 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:41.547013 27746 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:41.549456 27581 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:41.547926 27792 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:41.555112 27716 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:41.559661 27581 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.567404 27792 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:41.568655 27792 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.560575 27716 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:41.572062 27716 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.558187 27517 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:41.580054 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:41.586598 27587 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:41.588688 27517 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:41.589497 27517 consensus_queue.cc:1243] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:14:41.558840 27746 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.613824 27624 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.619477 27624 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:41.615330 27716 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.621781 27581 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:41.624943 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:41.627996 27760 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:41.628094 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:41.628608 27760 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:41.628886 27345 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:41.629451 27345 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.629839 27345 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.629572 27760 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:41.628966 25113 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:41.635803 27699 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:41.636471 27699 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:41.637446 27699 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:42.112211 23780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.010s
W20260709 12:14:42.200228 23780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.001s	sys 0.011s
W20260709 12:14:42.500624 27792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.011s	sys 0.000s
I20260709 12:14:42.479290 27716 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:42.568306 27792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.012s	sys 0.000s
I20260709 12:14:42.560801 27440 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:42.587492 27746 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.984s	user 0.005s	sys 0.007s
I20260709 12:14:42.582002 27716 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:42.591320 27805 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 12:14:42.611965 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.004s	sys 0.024s
W20260709 12:14:42.612387 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.004s	sys 0.024s
W20260709 12:14:42.615411 25113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.002s	sys 0.026s
W20260709 12:14:42.615921 25113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.002s	sys 0.026s
W20260709 12:14:42.614138 25112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.026s
I20260709 12:14:42.601636 27581 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:42.616813 25112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.000s	sys 0.026s
I20260709 12:14:42.628742 25175 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:14:42.634728 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.017s	sys 0.028s
W20260709 12:14:42.635106 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.017s	sys 0.028s
I20260709 12:14:42.610363 27716 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.655313 27716 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.627817 27519 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:42.648399 27699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.133s	sys 0.233s
I20260709 12:14:42.589651 27440 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.648123 27709 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:42.613241 27746 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.667292 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 896) took 1201 ms. Trace:
W20260709 12:14:42.660696 27699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.133s	sys 0.234s
I20260709 12:14:42.667855 25176 rpcz_store.cc:276] 0709 12:14:41.463498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.463662 (+   164us) service_pool.cc:224] Handling call
0709 12:14:41.464009 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:41.464408 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:41.464440 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:41.468033 (+  3593us) log.cc:844] Serialized 26 byte log entry
0709 12:14:41.557293 (+ 89260us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:41.557347 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.557424 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:42.665087 (+1107663us) raft_consensus.cc:1685] finished
0709 12:14:42.665113 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:42.665315 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7419,"thread_start_us":25119,"threads_started":1,"wal-append.queue_time_us":152572}
I20260709 12:14:42.659909 27716 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:42.684979 25169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 313) took 1754 ms (client timeout 1980 ms). Trace:
W20260709 12:14:42.685446 25169 rpcz_store.cc:269] 0709 12:14:40.928643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.929026 (+   383us) service_pool.cc:224] Handling call
0709 12:14:42.682789 (+1753763us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400614254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:42.682907 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641539840}
I20260709 12:14:42.659145 27581 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:14:42.662101 27519 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:42.686702 27803 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:14:42.704659 27693 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:42.705271 27693 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:42.685410 27716 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:42.764678 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.010s	sys 0.030s
W20260709 12:14:42.765496 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.010s	sys 0.030s
W20260709 12:14:42.771901 27345 consensus_meta.cc:287] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.142s	user 0.009s	sys 0.004s
I20260709 12:14:42.772668 27345 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.751660 27693 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.892161 27699 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.900954 27693 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:42.900759 27699 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.891949 27440 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.903090 27699 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.890476 27816 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:42.905575 27816 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.907199 27716 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.898233 27519 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.898710 27345 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.900027 27818 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.908912 27818 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.909376 27746 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:42.903357 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:42.911437 23778 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.886950 27760 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:42.919056 27813 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.919715 27813 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.921283 27816 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.918341 27729 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.922632 27813 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.924999 27729 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.927814 27818 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.931460 23778 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:42.933898 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 697) took 1450 ms. Trace:
I20260709 12:14:42.934973 23778 rpcz_store.cc:276] 0709 12:14:41.483368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.522303 (+ 38935us) service_pool.cc:224] Handling call
0709 12:14:42.933785 (+1411482us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:42.936821 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:42.938556 23778 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:42.933445 27729 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.950099 27784 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:42.951057 27818 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.953802 27818 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.954098 23778 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:42.954820 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 694) took 1472 ms. Trace:
I20260709 12:14:42.955197 23778 rpcz_store.cc:276] 0709 12:14:41.482573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.936554 (+1453981us) service_pool.cc:224] Handling call
0709 12:14:42.954757 (+ 18203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:42.956789 27818 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.957547 27729 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.957664 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 695) took 1474 ms. Trace:
I20260709 12:14:42.958151 23778 rpcz_store.cc:276] 0709 12:14:41.482814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.955931 (+1473117us) service_pool.cc:224] Handling call
0709 12:14:42.957101 (+  1170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:42.957348 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:42.957374 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:42.957414 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:42.957449 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.957504 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:42.957612 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:42.944589 27760 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.958952 27729 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.961037 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:42.961699 25169 raft_consensus.cc:606] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:42.961705 27729 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.963227 27729 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:42.963990 25172 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:42.970350 27440 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:42.971078 27440 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.972091 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 699) took 1451 ms. Trace:
I20260709 12:14:42.970598 27760 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.973416 27811 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:42.974165 27440 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:42.963744 27729 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.968200 27535 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.974702 27815 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.975200 27815 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.969239 27519 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.969578 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f333e0a484248ce88f0905e18500838"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:42.976226 25169 raft_consensus.cc:606] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:42.976109 27519 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.969806 27716 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.976964 27729 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.977319 27716 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.978053 27519 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.975076 27535 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.979300 27815 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:42.979360 27716 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:42.968696 27746 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.980921 27716 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:42.981926 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:42.982543 25176 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.982846 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:42.983107 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:42.983726 25174 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:42.972458 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2601) took 1685 ms. Trace:
I20260709 12:14:42.985826 23776 rpcz_store.cc:276] 0709 12:14:41.287440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.418445 (+131005us) service_pool.cc:224] Handling call
0709 12:14:41.478593 (+ 60148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:41.492479 (+ 13886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.972232 (+1479753us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b5400168350. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:42.972359 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99733888}
I20260709 12:14:42.980661 27746 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:42.985828 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:42.981520 27716 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:42.991689 27716 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:42.989948 27581 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:42.972496 23778 rpcz_store.cc:276] 0709 12:14:41.520313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.961832 (+1441519us) service_pool.cc:224] Handling call
0709 12:14:42.971529 (+  9697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:42.971742 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:42.971787 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:42.971832 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:42.971871 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.971923 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:42.972035 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:42.997591 27581 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:42.998081 27581 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.999135 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:43.001226 27535 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.005767 23778 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:43.005951 25176 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:43.006254 23778 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.006878 27818 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:43.007112 23778 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.007407 27818 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.008104 23778 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.009951 27818 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.013155 27581 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:43.014701 23782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.595s	user 0.055s	sys 0.083s
W20260709 12:14:43.015021 23782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.596s	user 0.055s	sys 0.083s
I20260709 12:14:43.017604 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:43.018369 25176 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.020850 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2599) took 1737 ms. Trace:
I20260709 12:14:43.021279 23777 rpcz_store.cc:276] 0709 12:14:41.283795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.413144 (+129349us) service_pool.cc:224] Handling call
0709 12:14:41.457598 (+ 44454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:41.458294 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.020625 (+1562331us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b540010b150. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:43.020783 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86348544}
I20260709 12:14:43.021665 27581 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.027173 25176 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:43.029408 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:43.030391 27581 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:43.031345 27581 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:43.022461 23778 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:43.032035 23778 pending_rounds.cc:85] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 8
I20260709 12:14:43.032415 23778 pending_rounds.cc:107] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:14:43.032874 23778 raft_consensus.cc:2889] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:43.032440 27581 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.034863 25174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:14:43.037145 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.51888e+06us
I20260709 12:14:43.038781 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:43.039773 25174 raft_consensus.cc:606] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:43.040295 25174 raft_consensus.cc:614] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:43.042510 27746 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:43.045794 27792 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:43.045847 27824 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.046576 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2573) took 1877 ms. Trace:
I20260709 12:14:43.046653 27824 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.047238 27792 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.047873 27792 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:43.048590 27826 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.021358 27617 raft_consensus.cc:993] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:43.049683 27617 raft_consensus.cc:1081] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:43.043339 27816 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.049119 27826 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.050185 27792 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.050938 27816 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.052553 27826 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.046067 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:43.053221 27816 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.053840 27816 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:43.054291 27816 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.054171 27792 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:43.055158 27792 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:43.049119 27824 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.055559 27792 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.054657 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:43.056430 27824 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:43.056499 27792 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.057036 27824 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.057638 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:43.057857 27792 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.058298 27824 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.059171 27826 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:43.059180 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:43.061307 23778 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:43.062888 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.055244 27816 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.066303 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.066998 23777 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:43.067333 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.055651 25174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:43.061412 27826 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.069291 25174 raft_consensus.cc:606] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:43.069486 27818 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:43.069988 27818 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.069814 27826 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.072408 27830 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:43.075775 23928 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:43.076237 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.076854 23785 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:43.078640 25176 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:43.079538 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:43.066941 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.080403 27792 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:43.059329 25175 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:43.082706 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:43.083487 23777 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:43.083885 23777 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.086118 23749 leader_election.cc:400] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7.
I20260709 12:14:43.081079 27792 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.087494 27822 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:43.088394 27617 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:43.089561 25114 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:43.057873 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.098994 23783 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 6: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:43.047574 23782 rpcz_store.cc:276] 0709 12:14:41.169210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.380054 (+210844us) service_pool.cc:224] Handling call
0709 12:14:41.384099 (+  4045us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:41.396208 (+ 12109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:41.396239 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:41.401923 (+  5684us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.015364 (+1613441us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:43.018338 (+  2974us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.046270 (+ 27932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.046344 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:43.046382 (+    38us) raft_consensus.cc:1685] finished
0709 12:14:43.046393 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:43.046515 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26290,"prepare.queue_time_us":2504,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":30978,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":3150,"thread_start_us":1595939,"threads_started":1,"wal-append.queue_time_us":1009658}
I20260709 12:14:43.091939 27693 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.093261 23749 leader_election.cc:403] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:14:43.100245 23783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 698) took 1616 ms (client timeout 1910 ms). Trace:
I20260709 12:14:42.978855 23900 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:43.102171 23931 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:43.081135 27807 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 12:14:43.070971 27818 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.096736 23777 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:43.100571 23783 rpcz_store.cc:269] 0709 12:14:41.502589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.521170 (+ 18581us) service_pool.cc:224] Handling call
0709 12:14:43.100168 (+1578998us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:43.108229 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 129452us
I20260709 12:14:43.101446 27815 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:43.111905 27784 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7.
I20260709 12:14:43.112494 27792 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.106181 27693 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.121618 27693 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.122195 27693 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.122643 27693 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.124565 23777 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.125375 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2616) took 2345 ms. Trace:
I20260709 12:14:43.125738 23784 rpcz_store.cc:276] 0709 12:14:40.788566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.911491 (+122925us) service_pool.cc:224] Handling call
0709 12:14:40.924870 (+ 13379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:41.099690 (+174820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:41.099726 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:41.115950 (+ 16224us) log.cc:844] Serialized 26 byte log entry
0709 12:14:41.271624 (+155674us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:41.298829 (+ 27205us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.299045 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.299124 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.387077 (+ 87953us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b1800400360. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:41.571985 (+184908us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b1800400360. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:43.125152 (+1553167us) raft_consensus.cc:1685] finished
0709 12:14:43.125173 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:43.125295 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7256,"prepare.queue_time_us":30615,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":214498,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":208,"spinlock_wait_cycles":25862656,"thread_start_us":267343,"threads_started":2,"wal-append.queue_time_us":200135,"wal-append.run_cpu_time_us":2012,"wal-append.run_wall_time_us":1002641}
I20260709 12:14:43.128453 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2615) took 2349 ms. Trace:
W20260709 12:14:43.153481 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 67514.3us
I20260709 12:14:43.127395 27825 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.159837 27818 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:43.160472 27818 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.162524 27693 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.112221 27815 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.128880 23780 rpcz_store.cc:276] 0709 12:14:40.779402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.792361 (+ 12959us) service_pool.cc:224] Handling call
0709 12:14:40.823633 (+ 31272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:40.859370 (+ 35737us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:41.109901 (+250531us) log.cc:844] Serialized 14 byte log entry
0709 12:14:41.586504 (+476603us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180015b620. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:42.765916 (+1179412us) log.cc:844] Serialized 14 byte log entry
0709 12:14:42.766115 (+   199us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:42.766560 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:14:42.766691 (+   131us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:42.766851 (+   160us) log.cc:844] Serialized 14 byte log entry
0709 12:14:42.949227 (+182376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:42.953515 (+  4288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:43.080597 (+127082us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:43.128209 (+ 47612us) raft_consensus.cc:1685] finished
0709 12:14:43.128232 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:43.128387 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100038,"prepare.queue_time_us":438702,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":100,"raft.queue_time_us":801540,"raft.run_cpu_time_us":753,"raft.run_wall_time_us":142706,"spinlock_wait_cycles":9106048,"thread_start_us":1437243,"threads_started":5,"wal-append.queue_time_us":1488587}
W20260709 12:14:43.155859 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46744.2us
I20260709 12:14:43.130849 27517 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:43.167459 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.171361 25175 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:43.158706 27709 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.175036 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.176201 23928 raft_consensus.cc:2410] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:14:43.174214 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.177932 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.178295 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.178943 23927 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:43.179751 23932 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:43.179975 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.180222 25176 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:43.181485 23929 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:43.182159 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:43.169472 27815 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:43.185314 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.185672 23925 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:43.185946 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:43.185981 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.188067 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.176040 27833 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.192284 27833 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.196420 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:43.197166 25174 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:43.197479 25174 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:43.210161 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.198316 25174 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.210945 23925 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:43.212112 25174 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.166735 27825 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.220824 25174 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:43.228145 23928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:14:43.228962 23928 raft_consensus.cc:606] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:43.229373 23928 raft_consensus.cc:614] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:43.206090 27693 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.180045 27709 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.214006 27818 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.206092 27831 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.235944 25175 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:43.236141 27824 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:43.273658 27834 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:14:43.280454 27829 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:43.240471 27818 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.288112 25177 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:14:43.290887 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 322) took 2094 ms. Trace:
I20260709 12:14:43.297278 25177 rpcz_store.cc:276] 0709 12:14:41.196058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.196243 (+   185us) service_pool.cc:224] Handling call
0709 12:14:43.268284 (+2072041us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5800198f98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:43.268518 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.290586 (+ 22068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.290821 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36219136}
I20260709 12:14:43.235893 27693 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:43.307576 23751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 2.029s (SENT) [suppressed 40 similar messages]
I20260709 12:14:43.289333 27833 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.298738 23672 maintenance_manager.cc:419] P 8f6194fc84354daa9cd0b57de46d8e32: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033699
I20260709 12:14:43.288630 27837 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:43.314549 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:43.243311 27831 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.315778 25177 raft_consensus.cc:606] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:43.318084 25177 raft_consensus.cc:614] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:43.321043 27841 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.321647 27841 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.322510 27709 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.323836 27837 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.324653 27825 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.326160 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.326668 27842 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:43.326860 23782 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:14:43.327224 27842 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.326294 27837 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:43.311100 27828 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:14:43.287526 27824 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.298252 27818 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.330336 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:43.338376 27808 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.686s	user 0.003s	sys 0.002s
I20260709 12:14:43.318048 27833 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.343779 27837 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.343724 27709 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.342248 27838 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.352298 27838 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.353475 27831 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.354672 27838 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.354959 27831 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.357079 27693 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.355479 27831 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.358196 27709 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.360460 27831 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.361171 27831 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.355424 27758 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.361665 27831 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.362214 27758 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.364531 27758 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.360562 27709 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.368317 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.379400 27831 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.380061 27818 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.381053 27841 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.386973 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.387629 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.387712 23782 raft_consensus.cc:2376] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10. Re-sending same reply.
I20260709 12:14:43.388571 23778 raft_consensus.cc:2376] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9. Re-sending same reply.
I20260709 12:14:43.389999 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.390594 27835 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.392330 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.392978 23778 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:43.394368 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.394611 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.394973 23778 raft_consensus.cc:2376] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10. Re-sending same reply.
I20260709 12:14:43.395287 23782 raft_consensus.cc:2368] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
I20260709 12:14:43.396549 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.398648 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.404721 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.405498 23781 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:43.407279 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66947.5us
I20260709 12:14:43.408524 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:43.348451 27833 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.414278 23781 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:43.373157 27837 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.416445 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.415094 27535 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.426450 27535 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.391207 27835 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.425812 23781 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:43.415784 27824 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.418404 27519 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.438936 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.439818 25169 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:43.440678 27535 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.441231 27535 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.441238 27709 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.441954 27709 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.445782 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:43.446471 25169 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:43.447441 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:43.451345 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.448917 23749 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:14:43.452076 23825 connection.cc:256] RPC call timeout handler was delayed by -1.9084519900000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 12:14:43.452658 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.404s (SENT)
I20260709 12:14:43.453042 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:43.454008 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.427809 27716 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.492095 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.494990 27716 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.496935 27840 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:43.427520 27838 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:14:43.533177 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 84283.3us
I20260709 12:14:43.538408 27830 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:14:43.541528 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88896us
W20260709 12:14:43.542229 23825 connection.cc:256] RPC call timeout handler was delayed by -1.7956662329997926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 12:14:43.542546 23825 connection.cc:256] RPC call timeout handler was delayed by -1.7884662329997809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 12:14:43.441936 27535 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.557674 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:43.540385 27846 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:14:43.455938 27824 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
W20260709 12:14:43.564453 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 139027us
I20260709 12:14:43.550323 23781 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:43.565824 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:43.537735 27816 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:43.450989 27519 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.569228 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:43.569589 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.570554 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:43.588762 23781 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.563200 27716 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:43.587123 27840 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:43.614413 23927 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:43.544605 27844 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:14:43.630512 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:43.632408 23825 connection.cc:256] RPC call timeout handler was delayed by -1.6127527510000164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3235s for a response
W20260709 12:14:43.634272 23825 connection.cc:256] RPC call timeout handler was delayed by -1.574487703999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:14:43.634869 23825 connection.cc:256] RPC call timeout handler was delayed by -1.4471592619997864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
W20260709 12:14:43.623342 27846 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:14:43.633812 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.676890 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:43.678087 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:43.701411 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 166933us
I20260709 12:14:43.706456 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:43.727108 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.728160 23781 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:43.757122 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 327500us
W20260709 12:14:43.762766 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 195078us
W20260709 12:14:43.728367 27815 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.005s
I20260709 12:14:43.697664 27833 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.762292 27860 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.774837 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:43.778755 23858 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.775743 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2610) took 2997 ms. Trace:
I20260709 12:14:43.776408 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:43.776880 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.782552 23855 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:43.782714 23856 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.783869 23854 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:43.784233 23854 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:43.785679 23781 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:43.786624 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.787250 23781 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:43.762682 27843 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:43.760754 27838 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:43.836141 27838 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:43.837174 27838 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.773952 27824 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:43.779875 23779 rpcz_store.cc:276] 0709 12:14:40.778067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.782459 (+  4392us) service_pool.cc:224] Handling call
0709 12:14:40.822484 (+ 40025us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:41.201410 (+378926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:41.201444 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:41.471476 (+270032us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:14:43.286287 (+1814811us) log.cc:844] Serialized 52 byte log entry
0709 12:14:43.339580 (+ 53293us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:43.340027 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.340377 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.340521 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.340581 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:43.775418 (+434837us) raft_consensus.cc:1685] finished
0709 12:14:43.775442 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:43.775669 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24725,"prepare.queue_time_us":286,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":277960,"raft.run_cpu_time_us":1980,"raft.run_wall_time_us":390842,"spinlock_wait_cycles":7113984,"thread_start_us":394193,"threads_started":2,"wal-append.queue_time_us":64673}
W20260709 12:14:43.841375 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43979.7us
I20260709 12:14:43.776315 27837 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.804529 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:43.776079 27709 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:43.842181 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:43.806939 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:43.846081 23855 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:43.784931 23854 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:43.847321 23854 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:43.850950 23858 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:43.775254 27858 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:43.855675 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:43.857944 27857 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
W20260709 12:14:43.858938 27859 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:14:43.778043 27815 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.874037 27843 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.914124 27866 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.914685 27866 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:43.904644 27440 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:43.842011 27824 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:43.913136 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:43.791747 27860 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.776476 27840 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:44.045884 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.447206 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.447621 27880 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.448104 23781 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:44.448171 27880 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:43.903393 27831 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:44.449597 27831 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.450282 23779 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:44.450605 27880 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.450884 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2650) took 1021 ms. Trace:
I20260709 12:14:44.115343 23854 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 12:14:43.933907 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 227542us
I20260709 12:14:44.117300 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:43.961863 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 119755us
W20260709 12:14:44.115206 23825 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.541s (SENT)
I20260709 12:14:44.453675 23854 pending_rounds.cc:85] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 5
I20260709 12:14:43.915300 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:43.916077 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.440686 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:44.456089 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 340899us
W20260709 12:14:44.446408 23856 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.663s	user 0.013s	sys 0.019s
I20260709 12:14:44.457710 23855 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:44.454145 23854 pending_rounds.cc:107] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:14:44.458768 23854 raft_consensus.cc:2889] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:44.457425 23856 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:44.456113 23858 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:44.448778 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2646) took 1021 ms. Trace:
I20260709 12:14:44.454123 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:44.477342 27879 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.451464 23779 rpcz_store.cc:276] 0709 12:14:43.429380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.853482 (+424102us) service_pool.cc:224] Handling call
0709 12:14:44.450822 (+597340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:44.478109 27879 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.479876 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.466689 23781 rpcz_store.cc:276] 0709 12:14:43.428041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.787849 (+359808us) service_pool.cc:224] Handling call
0709 12:14:44.448684 (+660835us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:44.484387 23852 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:44.480983 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:43.854082 27833 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:44.432380 27873 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.305s	user 0.000s	sys 0.000s
I20260709 12:14:43.888967 27816 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:14:44.126417 27854 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 12:14:44.061136 27857 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:14:44.488837 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.061738 27856 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.485267 23858 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:44.437664 27844 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.486667 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.438951 27815 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.440562 27838 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:44.392124 27862 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.441143 27853 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:44.439829 27865 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.916046 27867 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.639s
I20260709 12:14:44.040160 27535 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:44.451786 27519 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.130239 27858 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.969031 27837 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:43.915668 27843 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.601521 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.557036 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.606189 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.617971 25169 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:44.551398 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.601552 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.617450 25177 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:44.524133 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.619050 23859 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:44.601713 23933 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:44.501434 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.550614 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.549863 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:44.623235 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2648) took 1194 ms. Trace:
I20260709 12:14:44.623584 23779 rpcz_store.cc:276] 0709 12:14:43.428901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.486142 (+1057241us) service_pool.cc:224] Handling call
0709 12:14:44.623168 (+137026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:44.540411 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.526204 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.526599 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.625850 23854 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:44.601413 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:44.627887 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 171130us
I20260709 12:14:44.523218 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.558655 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2604) took 1171 ms. Trace:
W20260709 12:14:44.629093 23825 leader_election.cc:336] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.898s (SENT)
I20260709 12:14:44.629560 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:44.633291 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.446614 27835 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.634943 23858 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:44.634318 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.629230 23783 rpcz_store.cc:276] 0709 12:14:43.386907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.387025 (+   118us) service_pool.cc:224] Handling call
0709 12:14:43.408277 (+ 21252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.418529 (+ 10252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.418559 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.427131 (+  8572us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.558633 (+131502us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:43.584920 (+ 26287us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.848331 (+263411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.848417 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.558456 (+710039us) raft_consensus.cc:1685] finished
0709 12:14:44.558477 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.558596 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9881,"prepare.queue_time_us":19367,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":21124,"raft.queue_time_us":216352,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":41209,"spinlock_wait_cycles":5006080,"thread_start_us":394080,"threads_started":2,"wal-append.queue_time_us":200003}
I20260709 12:14:44.629489 23857 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:44.561354 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:44.479507 27864 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.649710 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.639545 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.650346 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.646898 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.651837 23932 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:44.653118 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.487923 27833 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.649323 23853 raft_consensus.cc:2376] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10. Re-sending same reply.
I20260709 12:14:44.562150 27757 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.655844 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.658623 27884 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:44.634593 27862 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:44.656000 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.656601 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.659770 23859 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:44.659897 23855 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:44.660661 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.661368 27876 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:44.662030 27862 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.661743 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.653789 23928 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:44.663810 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.664528 23932 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:14:44.635099 27856 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.665993 27868 consensus_queue.cc:1059] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:44.667536 27883 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:44.668035 27883 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:44.634377 27844 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:44.602473 27875 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:44.661125 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:44.660622 23925 raft_consensus.cc:1240] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:44.670320 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:44.656342 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.657097 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:44.662101 27757 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.676322 27882 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:44.676841 27882 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.678297 27757 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.634840 27853 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.663931 23781 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:44.679297 27875 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:44.679792 27885 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.680366 27885 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.676976 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.656791 27880 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:44.674660 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:44.659271 27884 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.682768 27885 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.679788 27875 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:44.674817 23857 raft_consensus.cc:2376] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10. Re-sending same reply.
I20260709 12:14:44.525043 27648 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.668995 27883 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.680336 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2656) took 1242 ms. Trace:
I20260709 12:14:44.657801 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:44.701556 25177 raft_consensus.cc:2410] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:14:44.700820 23781 rpcz_store.cc:276] 0709 12:14:43.437601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.635122 (+1197521us) service_pool.cc:224] Handling call
0709 12:14:44.680164 (+ 45042us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5400093654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:44.680268 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31063296}
I20260709 12:14:44.703445 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.704851 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.705542 23781 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:44.706465 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.707101 23781 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:44.710372 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:44.685518 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56435.8us
W20260709 12:14:44.711390 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.906s (SENT)
I20260709 12:14:44.711879 23825 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:44.689013 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:44.690430 23933 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:14:44.693311 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2639) took 1291 ms. Trace:
I20260709 12:14:44.694129 27880 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:44.694789 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44898.1us
I20260709 12:14:44.670183 27833 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:44.675333 27864 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:44.695955 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.696175 23853 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:44.635262 27858 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:44.667901 27856 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.697085 27862 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:44.697211 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 713) took 1412 ms. Trace:
I20260709 12:14:44.677138 27844 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:44.743789 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 289729us
I20260709 12:14:44.634711 27865 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.697772 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:44.746723 23779 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:44.698614 27884 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:44.699010 25172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 319) took 3576 ms (client timeout 1521 ms). Trace:
I20260709 12:14:44.700115 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2640) took 1297 ms. Trace:
I20260709 12:14:44.559923 27866 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:44.685490 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:44.556375 27816 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.733237 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:44.661803 27876 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.733693 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:44.739046 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.964s (SENT)
W20260709 12:14:44.761898 23826 connection.cc:256] RPC call timeout handler was delayed by -1.1140900579998743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:14:44.635383 27837 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.699334 27879 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.732452 27792 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:44.743106 23784 rpcz_store.cc:276] 0709 12:14:43.284744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.284861 (+   117us) service_pool.cc:224] Handling call
0709 12:14:43.285167 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.341042 (+ 55875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.341073 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:43.341111 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:43.341139 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.696965 (+1355826us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b540010c554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:44.697012 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.697149 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128059776}
I20260709 12:14:44.765328 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:44.697638 27882 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:44.732993 27831 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.713267 23785 rpcz_store.cc:276] 0709 12:14:43.402093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.402235 (+   142us) service_pool.cc:224] Handling call
0709 12:14:43.402604 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.432257 (+ 29653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.432294 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.437738 (+  5444us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.726136 (+288398us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:43.768547 (+ 42411us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.773915 (+  5368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.683517 (+909602us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5400370fd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:44.683568 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.683635 (+    67us) raft_consensus.cc:1685] finished
0709 12:14:44.693063 (+  9428us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.693244 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90829,"prepare.queue_time_us":119667,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":102192,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":28124,"spinlock_wait_cycles":37485824,"thread_start_us":192526,"threads_started":1,"wal-append.queue_time_us":419509}
I20260709 12:14:44.737381 27883 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:44.749538 23859 raft_consensus.cc:2410] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:14:44.563323 27824 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.714747 23781 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:44.733922 27880 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.737049 27867 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:44.765664 27876 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:44.737041 23925 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
W20260709 12:14:44.762742 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 3.235s (SENT)
I20260709 12:14:44.740979 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2637) took 1341 ms. Trace:
W20260709 12:14:44.749775 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 91439.2us
I20260709 12:14:44.737156 27835 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:14:44.797636 27877 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:44.764930 27866 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:44.747169 27862 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.245225 27866 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.246656 27862 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.760926 27856 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.248003 27856 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.250182 27856 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.773043 27844 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:46.250840 27844 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:46.251271 27844 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:44.808895 27824 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:44.790904 27880 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:44.823304 27865 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.831845 27860 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.833412 27858 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.833787 27875 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:44.850914 27882 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.255532 27882 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:46.255694 27875 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:44.882992 27833 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.258368 27833 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.919872 27876 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:46.260653 27876 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.928021 27864 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:44.947045 27861 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.821s	user 0.002s	sys 0.006s
W20260709 12:14:44.961826 27826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.875s	user 0.016s	sys 0.016s
W20260709 12:14:46.265678 27826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.193s	user 0.017s	sys 0.016s
I20260709 12:14:46.266567 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2638) took 2865 ms. Trace:
I20260709 12:14:44.935467 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 167) took 5658 ms. Trace:
I20260709 12:14:45.175792 27837 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:44.704419 27648 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.743989 27760 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.122794 27648 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:44.864897 27784 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:44.751930 27631 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:46.010655 27907 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.154232 27907 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:46.035247 27908 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.155946 27908 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:46.035854 27910 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.080327 27911 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.157747 27910 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.157799 27911 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:46.210274 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.775202 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:44.911868 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:46.020529 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0027697870000338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
I20260709 12:14:44.748745 23776 rpcz_store.cc:276] 0709 12:14:43.403052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.403194 (+   142us) service_pool.cc:224] Handling call
0709 12:14:43.403564 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.413239 (+  9675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.413268 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.413785 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.538622 (+124837us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:43.538952 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.539121 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.539196 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.699915 (+1160719us) raft_consensus.cc:1685] finished
0709 12:14:44.699935 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.700056 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9251,"prepare.queue_time_us":1050,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":202139,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":30317,"spinlock_wait_cycles":28357504,"thread_start_us":124458,"threads_started":1,"wal-append.queue_time_us":162985}
I20260709 12:14:44.888165 23779 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:47.205703 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 721) took 2756 ms (client timeout 1524 ms). Trace:
W20260709 12:14:47.206123 23779 rpcz_store.cc:269] 0709 12:14:44.449420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.651430 (+202010us) service_pool.cc:224] Handling call
0709 12:14:47.205581 (+2554151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:46.213222 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 712) took 3019 ms. Trace:
I20260709 12:14:44.956549 23782 rpcz_store.cc:276] 0709 12:14:43.399954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.400067 (+   113us) service_pool.cc:224] Handling call
0709 12:14:43.400420 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.416067 (+ 15647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.416099 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.416476 (+   377us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.595661 (+179185us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:43.862117 (+266456us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.862342 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.862414 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.740762 (+878348us) raft_consensus.cc:1685] finished
0709 12:14:44.740782 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.740917 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15999,"prepare.queue_time_us":176516,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":200706,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":51156,"thread_start_us":391113,"threads_started":2,"wal-append.queue_time_us":327650}
I20260709 12:14:44.938179 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2643) took 1514 ms. Trace:
I20260709 12:14:46.077047 23777 pending_rounds.cc:85] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 3
I20260709 12:14:47.216606 23777 pending_rounds.cc:107] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:14:47.216917 23777 raft_consensus.cc:2889] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:46.067466 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2645) took 2642 ms. Trace:
I20260709 12:14:47.218732 23781 rpcz_store.cc:276] 0709 12:14:43.427345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.066576 (+2639231us) service_pool.cc:224] Handling call
0709 12:14:46.066932 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.067126 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.067150 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:46.067193 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:46.067231 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.067281 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.067400 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:44.768263 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:44.816013 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:47.220712 23784 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
W20260709 12:14:44.814761 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.009s	sys 0.014s
W20260709 12:14:44.929637 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:44.956879 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 194164us
I20260709 12:14:47.226037 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:47.226699 23857 raft_consensus.cc:606] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:44.821535 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1245) took 1045 ms. Trace:
I20260709 12:14:47.227712 23851 rpcz_store.cc:276] 0709 12:14:43.775694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.482630 (+706936us) service_pool.cc:224] Handling call
0709 12:14:44.482960 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:44.556352 (+ 73392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.556383 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:44.556871 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:14:44.705760 (+148889us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:44.705806 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.705875 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.821286 (+115411us) raft_consensus.cc:1685] finished
0709 12:14:44.821308 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.821464 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2877,"raft.queue_time_us":73360,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":148492,"threads_started":1,"wal-append.queue_time_us":196663,"wal-append.run_cpu_time_us":2132,"wal-append.run_wall_time_us":1057837}
I20260709 12:14:46.277803 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1268) took 1726 ms. Trace:
I20260709 12:14:47.230136 23859 rpcz_store.cc:276] 0709 12:14:44.551677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.660416 (+108739us) service_pool.cc:224] Handling call
0709 12:14:46.277497 (+1617081us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b54002d41d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:46.277695 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202172288}
I20260709 12:14:44.779165 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:46.222824 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080029cb40 after lost signal to thread 22632
W20260709 12:14:47.234887 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080031c020 after lost signal to thread 23717
I20260709 12:14:44.754042 23899 leader_election.cc:400] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
W20260709 12:14:46.172513 23900 connection.cc:256] RPC call timeout handler was delayed by -1.8432237399999849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 12:14:47.237386 23900 connection.cc:256] RPC call timeout handler was delayed by -1.7496237399998336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 12:14:47.237695 23900 connection.cc:256] RPC call timeout handler was delayed by -1.6301251309998861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 12:14:44.823925 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1159) took 1364 ms. Trace:
I20260709 12:14:47.238611 23927 rpcz_store.cc:276] 0709 12:14:43.459879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.491753 (+ 31874us) service_pool.cc:224] Handling call
0709 12:14:44.823729 (+1331976us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54000f91d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:44.823836 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279363456}
W20260709 12:14:47.239984 23927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2623) took 2326 ms (client timeout 1810 ms). Trace:
W20260709 12:14:47.240391 23927 rpcz_store.cc:269] 0709 12:14:44.913583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.239729 (+2326146us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.239875 (+   146us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:14:46.157791 23926 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.417s	user 0.027s	sys 0.022s
I20260709 12:14:44.974474 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.244989 23928 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:47.245525 23928 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1171) took 2339 ms (client timeout 1579 ms). Trace:
I20260709 12:14:46.198377 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:47.246976 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2630) took 2330 ms (client timeout 1637 ms). Trace:
W20260709 12:14:47.247401 23929 rpcz_store.cc:269] 0709 12:14:44.916816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.198030 (+1281214us) service_pool.cc:224] Handling call
0709 12:14:47.246886 (+1048856us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:47.248436 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2631) took 2331 ms (client timeout 1882 ms). Trace:
W20260709 12:14:47.248808 23929 rpcz_store.cc:269] 0709 12:14:44.917197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.248228 (+2331031us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.248370 (+   142us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:14:47.249607 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2629) took 2333 ms (client timeout 1928 ms). Trace:
W20260709 12:14:47.249922 23929 rpcz_store.cc:269] 0709 12:14:44.916410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.249444 (+2333034us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.249558 (+   114us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:14:47.250630 23929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2625) took 2336 ms (client timeout 1966 ms). Trace:
I20260709 12:14:46.220795 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.251683 27932 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.285917 23932 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:47.253562 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2620) took 2512 ms. Trace:
I20260709 12:14:47.253901 23932 rpcz_store.cc:276] 0709 12:14:44.741573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.285365 (+1543792us) service_pool.cc:224] Handling call
0709 12:14:46.285729 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.253260 (+967531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.253462 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:47.254994 23932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2622) took 2341 ms (client timeout 2019 ms). Trace:
W20260709 12:14:46.025154 23933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.279s	user 0.027s	sys 0.027s
W20260709 12:14:47.257448 23933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.511s	user 0.028s	sys 0.028s
I20260709 12:14:47.258272 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2619) took 2555 ms. Trace:
I20260709 12:14:46.209677 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:47.264659 23934 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2621) took 2352 ms (client timeout 1686 ms). Trace:
W20260709 12:14:47.265070 23934 rpcz_store.cc:269] 0709 12:14:44.912661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.209372 (+1296711us) service_pool.cc:224] Handling call
0709 12:14:47.264571 (+1055199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.266150 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.266799 23934 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
W20260709 12:14:46.268357 25113 connection.cc:256] RPC call timeout handler was delayed by -1.4332261990000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 12:14:47.268577 25113 connection.cc:256] RPC call timeout handler was delayed by -1.4273090790002243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 12:14:47.268924 25113 connection.cc:256] RPC call timeout handler was delayed by -1.4069737410000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 12:14:47.269178 25113 connection.cc:256] RPC call timeout handler was delayed by -1.3613855520002289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 12:14:47.269433 25113 connection.cc:256] RPC call timeout handler was delayed by -1.3431479129999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 12:14:47.269667 25113 connection.cc:256] RPC call timeout handler was delayed by -1.2926737410000442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
W20260709 12:14:47.270097 25113 connection.cc:256] RPC call timeout handler was delayed by -1.2567479130000265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 12:14:47.270370 25113 connection.cc:256] RPC call timeout handler was delayed by -1.2068090790000952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:14:47.270743 25113 connection.cc:256] RPC call timeout handler was delayed by -1.0039954070000476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260709 12:14:47.271998 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:46.154862 25119 connection.cc:256] RPC call timeout handler was delayed by -1.385697435000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
W20260709 12:14:47.273221 25119 connection.cc:256] RPC call timeout handler was delayed by -1.3436637620000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 12:14:47.273648 25119 connection.cc:256] RPC call timeout handler was delayed by -1.300472794000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 12:14:47.273944 25119 connection.cc:256] RPC call timeout handler was delayed by -1.2820959110001695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 12:14:47.274209 25119 connection.cc:256] RPC call timeout handler was delayed by -1.2451211120001062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
W20260709 12:14:44.748603 25172 rpcz_store.cc:269] 0709 12:14:41.122843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.195751 (+ 72908us) service_pool.cc:224] Handling call
0709 12:14:44.698806 (+3503055us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54001e87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:44.698915 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3743123840}
W20260709 12:14:45.173360 25175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 328) took 1987 ms (client timeout 2019 ms). Trace:
W20260709 12:14:46.876493 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08002187c0 after lost signal to thread 22629
I20260709 12:14:46.966030 27918 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.966676 27919 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:46.986879 23680 rpcz_store.cc:276] 0709 12:14:39.280874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.286508 (+  5634us) service_pool.cc:224] Handling call
0709 12:14:39.570166 (+283658us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:44.935386 (+5365220us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:40.365563 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:40.390570 (+ 25007us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:40.390604 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:14:40.390619 (+    15us) tablet.cc:662] Decoding operations
0709 12:14:40.464286 (+ 73667us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:40.464346 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:40.464371 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:14:40.465058 (+   687us) tablet.cc:701] Row locks acquired
0709 12:14:40.465077 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:14:40.465244 (+   167us) write_op.cc:270] Start()
0709 12:14:40.465377 (+   133us) write_op.cc:276] Timestamp: P: 1783599280465180 usec, L: 0
0709 12:14:40.465426 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:14:40.493268 (+ 27842us) log.cc:844] Serialized 3570 byte log entry
0709 12:14:40.702726 (+209458us) op_driver.cc:464] REPLICATION: finished
0709 12:14:41.038968 (+336242us) write_op.cc:301] APPLY: starting
0709 12:14:41.075137 (+ 36169us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:41.118624 (+ 43487us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:41.134927 (+ 16303us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:41.142231 (+  7304us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:41.142466 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:14:41.142509 (+    43us) write_op.cc:312] APPLY: finished
0709 12:14:41.142793 (+   284us) log.cc:844] Serialized 71 byte log entry
0709 12:14:41.142973 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:41.143107 (+   134us) write_op.cc:454] Released schema lock
0709 12:14:41.143152 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:41.143552 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:41.143581 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":255,"mutex_wait_us":289210,"reported_tablets":15,"spinlock_wait_cycles":559795584,"child_traces":[["op",{"apply.queue_time_us":317798,"apply.run_cpu_time_us":3293,"apply.run_wall_time_us":120718,"num_ops":9,"prepare.queue_time_us":81775,"prepare.run_cpu_time_us":4362,"prepare.run_wall_time_us":153545,"replication_time_us":237149,"thread_start_us":194726,"threads_started":2}]]}
I20260709 12:14:47.004865 27920 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:47.101012 23671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.005s	sys 0.010s
I20260709 12:14:47.110328 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.111564 25170 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:47.112486 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.113200 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:46.269935 27844 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.258599 23933 rpcz_store.cc:276] 0709 12:14:44.703198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.713325 (+ 10127us) service_pool.cc:224] Handling call
0709 12:14:44.713631 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.713857 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:44.745771 (+ 31914us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.257905 (+2512134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.257964 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:47.258014 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.258078 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.258203 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":128019,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":27,"raft.queue_time_us":9876,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":2543369,"threads_started":2,"wal-append.queue_time_us":83133,"wal-append.run_cpu_time_us":1371,"wal-append.run_wall_time_us":1004591}
I20260709 12:14:46.248006 27866 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.279204 27920 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:46.103183 27693 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:47.285970 25119 connection.cc:256] RPC call timeout handler was delayed by -1.194723655999951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
I20260709 12:14:47.122623 27760 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:47.282219 25175 rpcz_store.cc:269] 0709 12:14:43.185601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.185726 (+   125us) service_pool.cc:224] Handling call
0709 12:14:45.173143 (+1987417us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b54002526d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:45.173291 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113540992}
I20260709 12:14:47.288691 27760 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:47.130591 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2633) took 2199 ms (client timeout 1774 ms). Trace:
I20260709 12:14:47.131325 27784 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.767575 27792 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.290460 27792 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:45.311357 27817 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.292016 27784 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.292457 27817 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.293341 23785 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:47.294677 27817 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:44.918454 27818 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.286381 27693 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:44.877808 27842 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:46.238554 27843 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:46.003232 27848 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 2.610s	user 0.016s	sys 0.017s
I20260709 12:14:47.300806 27926 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:46.434159 27853 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:47.295496 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2608) took 2674 ms (client timeout 2031 ms). Trace:
I20260709 12:14:47.011343 27877 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:47.150775 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.304378 27877 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.305068 23683 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 7 to 8, leader changed from <none> to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 8 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:47.153244 27631 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:47.255337 23932 rpcz_store.cc:269] 0709 12:14:44.913161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.254827 (+2341666us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.254938 (+   111us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:47.296518 27818 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.309278 27631 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.161705 27907 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.309757 27818 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.164428 27837 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:47.172451 27911 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:47.175921 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 2.40058e+06us
I20260709 12:14:47.179337 27910 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.180022 27908 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.186614 27928 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:47.209054 23780 rpcz_store.cc:276] 0709 12:14:43.193477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.193898 (+   421us) service_pool.cc:224] Handling call
0709 12:14:43.208783 (+ 14885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:44.856647 (+1647864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.856679 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:44.865413 (+  8734us) log.cc:844] Serialized 26 byte log entry
0709 12:14:44.920819 (+ 55406us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:44.921029 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.212810 (+1291781us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540010bb54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:46.212860 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:46.212914 (+    54us) raft_consensus.cc:1685] finished
0709 12:14:46.212925 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.213150 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3926,"raft.queue_time_us":3273,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"spinlock_wait_cycles":3788776192,"thread_start_us":50407,"threads_started":1,"wal-append.queue_time_us":57833,"wal-append.run_cpu_time_us":2674,"wal-append.run_wall_time_us":1002893}
I20260709 12:14:47.214371 23783 rpcz_store.cc:276] 0709 12:14:43.424225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.937265 (+1513040us) service_pool.cc:224] Handling call
0709 12:14:44.937662 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.937860 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.937886 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:44.937927 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:44.937964 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.938014 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.938115 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.306787 23683 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:47.214993 23749 connection.cc:256] RPC call timeout handler was delayed by -1.1386489339997752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 12:14:47.312492 23749 leader_election.cc:336] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Timed out: RequestConsensusVote RPC to 127.22.25.68:43683 timed out after 1.521s (SENT)
I20260709 12:14:47.312896 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:46.266971 23778 rpcz_store.cc:276] 0709 12:14:43.400955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.401065 (+   110us) service_pool.cc:224] Handling call
0709 12:14:43.401489 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:43.432369 (+ 30880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.432398 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:43.432864 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:14:43.577613 (+144749us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:43.791222 (+213609us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.791423 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.791500 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.946236 (+1154736us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:45.168000 (+221764us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:14:46.266329 (+1098329us) raft_consensus.cc:1685] finished
0709 12:14:46.266354 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:46.266504 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71313,"prepare.queue_time_us":47093,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":154162,"raft.run_cpu_time_us":13941,"raft.run_wall_time_us":1749553,"spinlock_wait_cycles":21358720,"thread_start_us":246246,"threads_started":2,"wal-append.queue_time_us":504702}
I20260709 12:14:47.318280 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 722) took 2391 ms. Trace:
W20260709 12:14:47.221331 23784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2606) took 2601 ms (client timeout 1905 ms). Trace:
W20260709 12:14:47.232592 23860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1248) took 3456 ms (client timeout 1803 ms). Trace:
I20260709 12:14:47.236891 23899 leader_election.cc:403] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:14:47.237731 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08001047a0 after lost signal to thread 23809
W20260709 12:14:47.238062 23900 connection.cc:256] RPC call timeout handler was delayed by -1.5930237400000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 12:14:47.319334 23900 connection.cc:256] RPC call timeout handler was delayed by -1.5761251309997988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 12:14:47.318701 23779 rpcz_store.cc:276] 0709 12:14:44.927173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.206936 (+2279763us) service_pool.cc:224] Handling call
0709 12:14:47.207243 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.310461 (+103218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.310490 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.310526 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:47.310554 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.318039 (+  7485us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540012cad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:47.318085 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.318216 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4210,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":89,"spinlock_wait_cycles":236750208}
W20260709 12:14:47.243906 23927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2632) took 2326 ms (client timeout 1862 ms). Trace:
W20260709 12:14:47.245793 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.288s	user 0.011s	sys 0.015s
W20260709 12:14:47.320575 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 2.57507e+06us
I20260709 12:14:47.320659 27908 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.321146 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.321280 27908 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.321768 23932 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
W20260709 12:14:47.322599 23860 rpcz_store.cc:269] 0709 12:14:43.776594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.763237 (+986643us) service_pool.cc:224] Handling call
0709 12:14:47.232487 (+2469250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.250953 27929 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:47.323796 23853 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:47.323927 27929 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:47.324649 23784 rpcz_store.cc:269] 0709 12:14:44.620918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.767969 (+147051us) service_pool.cc:224] Handling call
0709 12:14:47.221252 (+2453283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.323419 27908 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.252162 27931 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.326802 27931 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:44.926923 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.329177 27923 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
I20260709 12:14:47.330473 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:47.252067 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1172) took 2345 ms (client timeout 1711 ms). Trace:
W20260709 12:14:47.334587 23931 rpcz_store.cc:269] 0709 12:14:44.906219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.220509 (+1314290us) service_pool.cc:224] Handling call
0709 12:14:47.251954 (+1031445us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:47.277483 23926 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.539s	user 0.029s	sys 0.023s
W20260709 12:14:47.280861 23955 debug-util.cc:398] Leaking SignalData structure 0x7b0800106620 after lost signal to thread 22632
W20260709 12:14:47.337618 23955 debug-util.cc:398] Leaking SignalData structure 0x7b0800045660 after lost signal to thread 23717
I20260709 12:14:47.335578 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.272583 27648 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.267918 27864 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.340032 27864 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.340839 27931 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.342170 27864 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.281190 27919 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.343412 27929 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.344363 27919 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:47.282828 23671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.007s	sys 0.011s
I20260709 12:14:47.341635 23933 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:14:47.345165 23671 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.114s	user 0.011s	sys 0.011s
I20260709 12:14:47.345679 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:47.339705 23931 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:47.347308 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1173) took 2440 ms. Trace:
I20260709 12:14:47.282508 27918 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.280568 25174 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:47.347640 23931 rpcz_store.cc:276] 0709 12:14:44.906588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.339061 (+2432473us) service_pool.cc:224] Handling call
0709 12:14:47.339368 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.347015 (+  7647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.347245 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.348497 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:47.285598 27920 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.289393 27844 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.348835 23931 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:47.298118 27693 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.298885 25173 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:14:47.350867 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.351142 27918 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:47.280448 23928 rpcz_store.cc:269] 0709 12:14:44.905852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.974125 (+ 68273us) service_pool.cc:224] Handling call
0709 12:14:47.245408 (+2271283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.338881 27648 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.352296 25174 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:47.352734 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2627) took 2437 ms. Trace:
I20260709 12:14:47.353091 23928 rpcz_store.cc:276] 0709 12:14:44.915191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.351978 (+2436787us) service_pool.cc:224] Handling call
0709 12:14:47.352278 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.352446 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.352471 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.352511 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:47.352540 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.352586 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.352681 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.354008 27648 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:47.289350 23930 rpcz_store.cc:269] 0709 12:14:44.930724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.129885 (+2199161us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.130502 (+   617us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:46.266983 27880 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.356513 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.357257 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.357663 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:47.357971 23927 rpcz_store.cc:269] 0709 12:14:44.917573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.241061 (+2323488us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.243812 (+  2751us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:47.358901 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.359117 23927 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:47.359534 23859 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:47.359745 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2634) took 2428 ms. Trace:
I20260709 12:14:47.252286 27932 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.303220 27853 raft_consensus.cc:993] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:47.311154 27837 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.359938 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.360483 27853 raft_consensus.cc:1081] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:47.360605 23856 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:47.361152 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.361883 27837 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.362291 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.356056 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.362885 27932 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.311779 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.313019 23683 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.314226 27928 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.366791 27928 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:47.346626 25171 raft_consensus.cc:2368] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8.
I20260709 12:14:47.370048 25171 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:47.314891 27842 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:47.313787 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0360229949999393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30390000000000006s for a response
I20260709 12:14:47.316514 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.374078 25172 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:47.371537 25171 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:14:47.357760 27880 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.376101 25171 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:47.315240 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:47.299525 27843 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:47.319788 23900 connection.cc:256] RPC call timeout handler was delayed by -1.36262373999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 12:14:47.377255 23900 connection.cc:256] RPC call timeout handler was delayed by -1.345725130999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:14:47.377406 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 62249.2us
I20260709 12:14:44.955137 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:47.379474 23925 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1170) took 2474 ms (client timeout 1524 ms). Trace:
W20260709 12:14:47.379902 23925 rpcz_store.cc:269] 0709 12:14:44.905501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.954762 (+ 49261us) service_pool.cc:224] Handling call
0709 12:14:47.379384 (+2424622us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.377607 27843 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.382198 27843 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:47.321308 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:47.319144 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080032dae0 after lost signal to thread 23810
W20260709 12:14:47.387624 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66380.9us
I20260709 12:14:47.388599 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:47.354825 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2628) took 2439 ms. Trace:
I20260709 12:14:47.389813 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.390554 23857 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:47.390779 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.367676 23927 rpcz_store.cc:276] 0709 12:14:44.931008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.358647 (+2427639us) service_pool.cc:224] Handling call
0709 12:14:47.358943 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.359471 (+   528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.359680 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:47.392712 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800212280 after lost signal to thread 23884
I20260709 12:14:47.392616 27792 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:47.389178 23928 rpcz_store.cc:276] 0709 12:14:44.915536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.354007 (+2438471us) service_pool.cc:224] Handling call
0709 12:14:47.354351 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.354519 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.354543 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.354584 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:47.354612 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.354659 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.354769 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:47.393968 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf3c0 after lost signal to thread 23958
W20260709 12:14:47.394277 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08003ac1c0 after lost signal to thread 23959
I20260709 12:14:47.363353 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.331955 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:47.395455 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08003dde40 after lost signal to thread 25207
W20260709 12:14:47.286326 25119 connection.cc:256] RPC call timeout handler was delayed by -1.1218841209999937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:14:47.396075 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08003f3680 after lost signal to thread 25208
I20260709 12:14:47.396481 27925 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:47.395726 27581 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:47.398329 27581 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:47.398609 25119 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.910s (SENT)
I20260709 12:14:47.398825 27581 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:47.292816 27784 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.399494 27784 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.399904 27581 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.354822 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.402272 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.402843 23851 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:47.402905 25174 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:47.395987 27926 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:47.351163 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.404779 27784 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.367941 23683 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.409021 23683 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 8 to 9, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:47.395435 27928 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:47.372665 23749 connection.cc:256] RPC call timeout handler was delayed by -1.0033307169999262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 12:14:47.411662 27928 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.318713 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.375427 25172 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:47.377158 27842 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:47.250948 23929 rpcz_store.cc:269] 0709 12:14:44.914411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.250483 (+2336072us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:47.250581 (+    98us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:47.382489 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.382779 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.413277 27842 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.413764 27842 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:47.414556 23860 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:47.383140 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.384096 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.386353 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:47.377640 23900 connection.cc:256] RPC call timeout handler was delayed by -1.3203237399998216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 12:14:47.395135 27792 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:47.416956 23900 connection.cc:256] RPC call timeout handler was delayed by -1.3097251309998228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 12:14:47.417328 23900 connection.cc:256] RPC call timeout handler was delayed by -1.2897237399999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
W20260709 12:14:47.303148 23785 rpcz_store.cc:269] 0709 12:14:44.621520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.793078 (+171558us) service_pool.cc:224] Handling call
0709 12:14:47.295303 (+2502225us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b5400167954. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:47.295421 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165728000}
W20260709 12:14:47.417764 23900 connection.cc:256] RPC call timeout handler was delayed by -1.2645237399999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 12:14:47.338917 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08001c52a0 after lost signal to thread 23809
W20260709 12:14:47.418080 23900 connection.cc:256] RPC call timeout handler was delayed by -1.2258237399998961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260709 12:14:47.418471 23900 connection.cc:256] RPC call timeout handler was delayed by -1.1189251309997417s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 12:14:47.418589 23955 debug-util.cc:398] Leaking SignalData structure 0x7b080005d920 after lost signal to thread 23810
W20260709 12:14:47.419809 23955 debug-util.cc:398] Leaking SignalData structure 0x7b0800157a80 after lost signal to thread 23884
W20260709 12:14:47.420353 23955 debug-util.cc:398] Leaking SignalData structure 0x7b080004cd60 after lost signal to thread 23885
I20260709 12:14:47.318248 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.417246 27792 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:47.407848 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:47.422350 23934 raft_consensus.cc:2410] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:14:47.422883 25119 connection.cc:256] RPC call timeout handler was delayed by -1.0238648649999504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 12:14:47.404645 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:47.423856 27923 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:47.426184 27842 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.378147 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:47.426752 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:47.427533 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49460.9us
I20260709 12:14:47.428234 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.428668 27842 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:47.429147 27926 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.429569 27926 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:47.430229 27926 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.429975 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:47.430648 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6be0 after lost signal to thread 23958
W20260709 12:14:47.431238 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9440 after lost signal to thread 23959
W20260709 12:14:47.432088 23955 debug-util.cc:398] Leaking SignalData structure 0x7b0800389a60 after lost signal to thread 25172
W20260709 12:14:47.432709 23955 debug-util.cc:398] Leaking SignalData structure 0x7b080015dd60 after lost signal to thread 25207
W20260709 12:14:47.432974 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08002724e0 after lost signal to thread 25208
W20260709 12:14:47.433336 23955 debug-util.cc:398] Leaking SignalData structure 0x7b08003b56c0 after lost signal to thread 27648
W20260709 12:14:47.433661 23955 debug-util.cc:398] Leaking SignalData structure 0x7b0800351920 after lost signal to thread 27760
I20260709 12:14:47.417412 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:47.437287 27817 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:47.411278 23683 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.439898 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.444990 23857 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:47.446696 27931 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:47.438649 27926 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.429906 27932 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:14:47.471294 27926 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:47.473685 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43793.1us
I20260709 12:14:47.442093 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:47.448202 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.474773 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.407465 23851 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:47.449936 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2624) took 2536 ms. Trace:
I20260709 12:14:47.477393 23933 rpcz_store.cc:276] 0709 12:14:44.913966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.341039 (+2427073us) service_pool.cc:224] Handling call
0709 12:14:47.341341 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.449636 (+108295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.449866 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.455176 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2626) took 2540 ms. Trace:
I20260709 12:14:47.468535 27923 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:47.478344 23931 rpcz_store.cc:276] 0709 12:14:44.914838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.348401 (+2433563us) service_pool.cc:224] Handling call
0709 12:14:47.348670 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.454865 (+106195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.455105 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:47.479957 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.479825 27931 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:47.480734 27784 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:47.481101 27784 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.481451 27784 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:47.484045 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:47.486588 27931 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.486673 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:47.487092 27931 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:47.510923 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 91117.4us
I20260709 12:14:47.512537 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.429548 27842 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:47.523295 25114 connection.cc:256] RPC call timeout handler was delayed by -1.5186273879999135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
W20260709 12:14:47.528110 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 194626us
I20260709 12:14:47.476048 27926 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:47.512109 27932 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:47.476048 27928 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:47.529906 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 303307us
I20260709 12:14:47.532231 23851 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:14:47.532825 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46220.6us
I20260709 12:14:47.533257 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.515013 27931 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.555006 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:47.555634 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:47.520260 27842 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.564810 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.565110 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.566586 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:47.570947 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56427.5us
W20260709 12:14:47.572333 23824 connection.cc:256] RPC call timeout handler was delayed by -1.4921883940000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 12:14:47.572741 23824 connection.cc:256] RPC call timeout handler was delayed by -1.4921883940000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 12:14:47.572729 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:47.573244 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3805883940001422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 12:14:47.573558 23824 connection.cc:256] RPC call timeout handler was delayed by -1.3147401599999284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 12:14:47.573913 23824 connection.cc:256] RPC call timeout handler was delayed by -1.2570256199999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
W20260709 12:14:47.574182 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1991523440001401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 12:14:47.574450 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1924883940000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
W20260709 12:14:47.574775 23824 connection.cc:256] RPC call timeout handler was delayed by -1.181688394000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 12:14:47.575120 23824 connection.cc:256] RPC call timeout handler was delayed by -1.136829003999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 12:14:47.575457 23824 connection.cc:256] RPC call timeout handler was delayed by -1.136829003999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 12:14:47.575738 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1026290040001641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 12:14:47.576002 23824 connection.cc:256] RPC call timeout handler was delayed by -1.1026290040001641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 12:14:47.576282 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0528344970000489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 12:14:47.576557 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0454773660001138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
W20260709 12:14:47.576824 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0450329990001137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 12:14:47.577412 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.545872 27932 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.475169 27877 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:47.578004 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0367562960000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 12:14:47.591905 23852 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:47.604957 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.644354 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.605698 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:47.644747 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.606729 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.645140 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.645618 23927 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:47.645757 23853 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:14:47.646008 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.646167 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.605868 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.646684 23854 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:47.647927 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:47.594146 23751 connection.cc:256] RPC call timeout handler was delayed by -1.5988884799999141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.255s for a response
I20260709 12:14:47.608479 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:47.609004 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.600401 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:47.644740 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.551915 27844 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.552080 27818 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.646667 23851 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:47.653136 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.653293 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.652009 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.650296 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.654757 23857 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:47.654877 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 2 to 5, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:14:47.655961 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.654773 23931 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:47.656510 23857 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:47.612515 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:47.654266 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.660832 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.711180 23934 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:47.613317 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:47.666235 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.685456 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:47.627352 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.672480 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.659904 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:47.749883 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 194334us
I20260709 12:14:47.552671 27933 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:47.642980 27932 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:47.561338 27928 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.577073 27842 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:47.547637 27792 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 12:14:47.779289 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:47.569825 27920 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.226s
I20260709 12:14:47.779265 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.780395 23851 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:47.780027 23852 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:47.594101 27907 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:47.551683 27929 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:47.713248 23928 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:47.790915 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:47.825832 23779 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:47.546818 27923 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:47.578123 27925 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:47.788079 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:47.787071 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:47.782963 23776 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:47.788628 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2618) took 3087 ms. Trace:
W20260709 12:14:47.800066 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.802568 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:47.782282 25168 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:47.778455 25173 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:47.578474 27864 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
W20260709 12:14:47.808748 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 158482us
I20260709 12:14:47.670373 27818 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.797011 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.574635 27919 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.233s
I20260709 12:14:47.807826 27693 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:47.651357 27844 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.807426 27581 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:47.873590 27925 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.818756 27933 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.850412 23926 rpcz_store.cc:276] 0709 12:14:44.701268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.701398 (+   130us) service_pool.cc:224] Handling call
0709 12:14:44.703911 (+  2513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.704144 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:47.336074 (+2631930us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.788239 (+452165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.788297 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:47.788354 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.788435 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.788563 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124479,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":329277,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":116,"thread_start_us":3049638,"threads_started":3}
I20260709 12:14:47.879674 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:47.895876 27648 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:47.896344 27648 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.896718 27648 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:47.899497 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 322151us
I20260709 12:14:47.892596 23854 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:47.900358 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.901669 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:47.905059 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 304707us
I20260709 12:14:47.905917 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:47.790019 27963 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:47.848656 27951 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.783808 27962 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.838989 27935 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.919907 27911 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:14:47.930356 23855 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:47.893761 27943 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:14:47.859835 27693 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:47.847205 27929 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.941294 27910 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:47.940730 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:47.949703 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.950675 23859 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:47.952636 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:47.955534 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 306165us
I20260709 12:14:47.961119 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:47.962965 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.963888 23859 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:47.927490 27963 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:47.967873 25113 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Timed out: RequestConsensusVote RPC to 127.22.25.67:40463 timed out after 1.651s (SENT)
I20260709 12:14:47.935032 27918 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:47.937486 27947 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:14:47.935531 27853 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:47.974261 27581 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:47.952298 27880 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:47.935480 27951 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:47.942682 27693 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:47.960731 27837 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:47.977483 27929 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:47.980741 27866 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:47.981837 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.983270 27974 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:47.984133 27944 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:47.985402 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.986119 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.060386 23857 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:47.990932 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 720) took 3542 ms. Trace:
I20260709 12:14:47.991118 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:47.996649 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.014334 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.060818 23776 rpcz_store.cc:276] 0709 12:14:44.448636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.185719 (+2737083us) service_pool.cc:224] Handling call
0709 12:14:47.187251 (+  1532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.863749 (+676498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.982210 (+118461us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54003705d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:47.990852 (+  8642us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231280000}
I20260709 12:14:48.033311 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:14:48.061506 23859 raft_consensus.cc:2410] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:48.062047 23852 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:48.063345 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:14:48.070480 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 110677us
I20260709 12:14:48.037725 27792 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:48.078259 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.079521 23860 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.081302 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.082553 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.083163 23852 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:48.083779 23855 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:48.086771 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.087512 23856 raft_consensus.cc:2368] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 6. Current term is 7.
I20260709 12:14:48.087752 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.089097 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.084688 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.090200 23855 raft_consensus.cc:2410] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:14:48.094947 23825 connection.cc:256] RPC call timeout handler was delayed by -2.8224691400000665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 12:14:48.095937 23825 connection.cc:256] RPC call timeout handler was delayed by -2.7090691400001106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
I20260709 12:14:47.938800 27962 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:48.030370 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 135329us
I20260709 12:14:48.053212 27911 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:48.098858 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 158130us
I20260709 12:14:47.927837 27933 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:47.945096 27950 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.990422 27918 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:47.969559 27923 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:48.044958 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 246882us
I20260709 12:14:48.032657 27581 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.112084 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.115412 23860 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:48.093971 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 177538us
W20260709 12:14:48.035373 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 238441us
I20260709 12:14:48.100740 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:48.056978 27866 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:48.073140 27974 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:47.991374 27963 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.105346 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:48.053740 27968 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:48.110270 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:48.169653 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 64410.7us
I20260709 12:14:48.171866 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:48.172297 23926 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:14:48.173085 27941 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 12:14:47.338278 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.22.25.68:43683)
0709 12:14:47.933061 (+594783us) server_negotiation.cc:207] Beginning negotiation
0709 12:14:47.934995 (+  1934us) server_negotiation.cc:400] Waiting for connection header
0709 12:14:47.938763 (+  3768us) server_negotiation.cc:408] Connection header received
0709 12:14:47.972145 (+ 33382us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0709 12:14:47.972198 (+    53us) server_negotiation.cc:462] Received NEGOTIATE request from client
0709 12:14:47.972661 (+   463us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0709 12:14:47.972940 (+   279us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0709 12:14:48.046398 (+ 73458us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50258: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":490707,"thread_start_us":177381,"threads_started":1}
I20260709 12:14:48.174336 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.174975 23926 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:48.080005 27877 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:48.101629 27906 negotiation.cc:336] Failed RPC negotiation. Trace:
0709 12:14:44.935741 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.22.25.68:43683 (local address 127.0.0.1:50258)
0709 12:14:44.960428 (+ 24687us) negotiation.cc:107] Waiting for socket to connect
0709 12:14:44.960593 (+   165us) client_negotiation.cc:175] Beginning negotiation
0709 12:14:44.961409 (+   816us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0709 12:14:47.988079 (+3026670us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.22.25.68:43683: received 0 of 25 requested bytes
Metrics: {"client-negotiator.queue_time_us":19185,"thread_start_us":12450,"threads_started":1}
I20260709 12:14:48.092199 27928 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.114355 27961 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:48.130780 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08002028c0 after lost signal to thread 22629
I20260709 12:14:48.051710 23671 maintenance_manager.cc:643] P 8f6194fc84354daa9cd0b57de46d8e32: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 4.730s	user 0.178s	sys 0.042s Metrics: {"bytes_written":29219,"cfile_init":1,"dirs.queue_time_us":15262,"dirs.run_cpu_time_us":446,"dirs.run_wall_time_us":8798,"drs_written":1,"lbm_read_time_us":423,"lbm_reads_lt_1ms":4,"lbm_write_time_us":92205,"lbm_writes_10-100_ms":4,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":63,"thread_start_us":1122032,"threads_started":2,"wal-append.queue_time_us":72532,"wal-append.run_cpu_time_us":419,"wal-append.run_wall_time_us":1002103}
W20260709 12:14:48.181106 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800303040 after lost signal to thread 22632
W20260709 12:14:48.169859 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 69063.7us
W20260709 12:14:48.170956 27944 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:14:48.103343 27792 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:48.177918 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65682.3us
I20260709 12:14:48.111482 27929 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:48.235359 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:48.237432 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080003d180 after lost signal to thread 23717
I20260709 12:14:48.170617 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.106521 27950 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:48.175734 27911 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:48.244398 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.285130 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.285452 23672 maintenance_manager.cc:419] P 8f6194fc84354daa9cd0b57de46d8e32: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10882 bytes on disk
I20260709 12:14:48.285756 23928 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:48.286259 27842 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.170203 27968 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.182327 27961 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:48.248131 27966 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:14:48.111169 27918 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.249230 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.300297 23934 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.182358 27920 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:48.250437 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.250599 27760 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:48.184531 27817 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:48.283968 27911 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.254451 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.255139 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.348817 23933 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:48.255867 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.256043 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.256843 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.258535 27784 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.003s
I20260709 12:14:48.261554 27648 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.263782 27935 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:48.264065 27937 log.cc:927] Time spent T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:14:48.300117 27929 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:48.168303 27955 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:48.267360 27910 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:48.255376 23926 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:48.273746 27936 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:48.278294 27792 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:48.278800 27952 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.350s	user 0.004s	sys 0.000s
I20260709 12:14:48.178422 27974 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:48.282135 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080018b2c0 after lost signal to thread 23748
W20260709 12:14:48.391048 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800045160 after lost signal to thread 23751
W20260709 12:14:48.282748 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 47923us
I20260709 12:14:48.283869 27950 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.234830 27844 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.245177 27818 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.371553 27935 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.253815 23932 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:48.287695 27980 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:48.265376 27931 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.000s
I20260709 12:14:48.373546 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.301008 27968 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:48.301716 27920 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:48.304780 27817 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.383225 25172 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:48.379639 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.375061 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 203268us
I20260709 12:14:48.406760 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.307194 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.394243 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800203920 after lost signal to thread 23776
I20260709 12:14:48.377753 27866 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.399677 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.407874 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7080 after lost signal to thread 23779
W20260709 12:14:48.408223 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003461a0 after lost signal to thread 23782
W20260709 12:14:48.408421 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080003cd20 after lost signal to thread 23783
W20260709 12:14:48.408648 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800045ac0 after lost signal to thread 23784
W20260709 12:14:48.408855 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080004d440 after lost signal to thread 23785
W20260709 12:14:48.409415 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080031d6e0 after lost signal to thread 23809
W20260709 12:14:48.409746 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08002f91a0 after lost signal to thread 23810
W20260709 12:14:48.409987 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800332240 after lost signal to thread 23823
W20260709 12:14:48.410372 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800170b80 after lost signal to thread 23824
W20260709 12:14:48.410652 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1de0 after lost signal to thread 23825
W20260709 12:14:48.411322 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08002d8ec0 after lost signal to thread 23851
I20260709 12:14:48.411748 27817 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.374161 27987 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:14:48.407027 27920 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.413023 27987 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:48.413409 27987 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:48.407526 23930 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:48.396612 27973 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:14:48.418143 27973 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:48.418493 27973 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:48.418815 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2651) took 4987 ms. Trace:
I20260709 12:14:48.419235 27933 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:48.424280 27976 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:14:48.405090 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 928) took 1100 ms. Trace:
I20260709 12:14:48.425060 27961 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.425410 25175 rpcz_store.cc:276] 0709 12:14:47.304385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.304575 (+   190us) service_pool.cc:224] Handling call
0709 12:14:47.304949 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.305342 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.305371 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.305871 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:14:47.519072 (+213201us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:47.519113 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.530993 (+ 11880us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.404796 (+873803us) raft_consensus.cc:1685] finished
0709 12:14:48.404820 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.404987 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4552,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":212476,"threads_started":1,"wal-append.queue_time_us":645446}
I20260709 12:14:48.387280 27792 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.427374 27842 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.428303 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.429419 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.416566 27962 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.432184 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.436021 27760 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:48.436776 27648 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.402781 23934 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:48.382091 27963 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.379467 27910 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:48.439778 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.401391 27980 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.440949 27980 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:48.441615 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.442443 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.442917 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.443339 23860 raft_consensus.cc:2410] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:48.440305 27910 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.443652 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.395787 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.443110 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.444407 27910 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:48.444734 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 49056.1us
I20260709 12:14:48.444885 23858 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:48.445292 23856 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:48.445694 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.364027 27784 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.447006 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.419210 23777 rpcz_store.cc:276] 0709 12:14:43.431257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.076396 (+2645139us) service_pool.cc:224] Handling call
0709 12:14:46.076806 (+   410us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:47.510104 (+1433298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.510145 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:47.510932 (+   787us) log.cc:844] Serialized 78 byte log entry
0709 12:14:47.545634 (+ 34702us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:47.862298 (+316664us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.862641 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.959200 (+ 96559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.959677 (+   477us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.278153 (+318476us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:48.418568 (+140415us) raft_consensus.cc:1685] finished
0709 12:14:48.418590 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.418753 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":540501,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":518795,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":20976,"spinlock_wait_cycles":8586112,"thread_start_us":692177,"threads_started":4,"wal-append.queue_time_us":313815}
I20260709 12:14:48.432964 23930 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.447989 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.447774 27907 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:48.443027 27963 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.405278 27931 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.449561 27784 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.450706 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.447835 27843 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:48.451385 27843 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:48.451789 27843 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:48.440258 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.453944 27693 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.448747 27853 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:48.456838 27932 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.002s
I20260709 12:14:48.457329 27932 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.450800 27931 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.453997 27866 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:48.463528 27648 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:14:48.459910 27951 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.466210 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:48.463577 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080021c3c0 after lost signal to thread 23860
W20260709 12:14:48.477560 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080038b200 after lost signal to thread 23884
I20260709 12:14:48.473973 27963 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:14:48.438280 27581 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.479048 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.478931 27907 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:48.480136 27991 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:48.465085 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:48.485766 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.485944 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.486397 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.486449 23931 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:48.486709 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.487139 23929 raft_consensus.cc:2376] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7. Re-sending same reply.
I20260709 12:14:48.485246 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.487808 23932 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:48.488279 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.488461 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.478104 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1f20 after lost signal to thread 23908
I20260709 12:14:48.472355 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:48.481323 27960 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:14:48.488778 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.465840 23928 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:48.489615 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080039f580 after lost signal to thread 23925
I20260709 12:14:48.490314 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.493294 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003a37c0 after lost signal to thread 23927
W20260709 12:14:48.493729 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080037fc80 after lost signal to thread 23928
W20260709 12:14:48.494504 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800409da0 after lost signal to thread 23958
W20260709 12:14:48.495013 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800409dc0 after lost signal to thread 23959
I20260709 12:14:48.490306 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.481168 27581 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.490964 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.494922 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.492596 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.497982 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1aa0 after lost signal to thread 25114
I20260709 12:14:48.499923 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.497641 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.502107 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:48.496263 27631 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:48.503981 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.506249 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.502317 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08001da160 after lost signal to thread 25172
I20260709 12:14:48.511145 27991 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:48.511669 27631 raft_consensus.cc:2764] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:14:48.505016 27907 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:48.505244 27853 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:48.513110 27853 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.513501 27853 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:48.513674 27971 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.266s	user 0.010s	sys 0.000s
I20260709 12:14:48.514619 27693 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.517171 27817 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.517993 27920 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.519953 27995 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:48.520797 27908 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:48.521203 27908 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:48.521790 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49525.6us
I20260709 12:14:48.509905 27967 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:48.522125 27908 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.522917 27967 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:48.523140 27970 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:14:48.520969 27911 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.521253 27961 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:48.523797 27963 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:48.524252 27963 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.524523 27843 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.526023 27951 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:48.527079 27946 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:48.523434 27967 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.511111 27918 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.508065 27989 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:48.526988 27961 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:48.528975 27994 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:48.524607 27963 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:48.521996 27982 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.275s	user 0.000s	sys 0.002s
W20260709 12:14:48.531049 27986 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 12:14:48.534140 27843 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:48.547104 27994 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:48.552517 27918 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:48.553620 27908 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.496903 27792 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.561233 27792 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:48.536904 27920 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.561561 27853 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:48.508708 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080036c160 after lost signal to thread 25175
I20260709 12:14:48.537002 27817 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.551885 27961 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:48.562875 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003ba7e0 after lost signal to thread 25207
W20260709 12:14:48.563144 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003fc620 after lost signal to thread 25208
I20260709 12:14:48.563242 27961 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:48.535442 27911 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:48.563535 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800392440 after lost signal to thread 27792
W20260709 12:14:48.563865 25204 debug-util.cc:398] Leaking SignalData structure 0x7b08003fc400 after lost signal to thread 27817
W20260709 12:14:48.564178 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800248600 after lost signal to thread 27818
I20260709 12:14:48.564651 27963 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.566737 27963 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.459497 27932 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:48.601740 23668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 422609us
I20260709 12:14:48.601739 23671 maintenance_manager.cc:643] P 8f6194fc84354daa9cd0b57de46d8e32: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.314s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":389,"lbm_reads_lt_1ms":4}
W20260709 12:14:48.612993 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 442310us
W20260709 12:14:48.613238 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.640779 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:48.642313 23784 raft_consensus.cc:2376] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8. Re-sending same reply.
I20260709 12:14:48.646292 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:48.640802 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:48.648710 23776 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:48.642251 23777 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:48.665279 23776 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
W20260709 12:14:48.668398 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:48.669782 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 165744us
I20260709 12:14:48.680297 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.682708 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.684567 23851 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:48.684823 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:48.684636 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.688644 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.690378 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.691915 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.692406 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1292) took 1042 ms. Trace:
I20260709 12:14:48.692813 23853 rpcz_store.cc:276] 0709 12:14:47.649751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.805406 (+155655us) service_pool.cc:224] Handling call
0709 12:14:47.805729 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.822950 (+ 17221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.822981 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.828122 (+  5141us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.049678 (+221556us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:48.049723 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.049874 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.522537 (+472663us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:48.691811 (+169274us) raft_consensus.cc:1685] finished
0709 12:14:48.691849 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.692251 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18487,"raft.queue_time_us":18652,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":97839872,"thread_start_us":221102,"threads_started":1,"wal-append.queue_time_us":412899}
I20260709 12:14:48.695498 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.697388 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.698697 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.693396 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.700045 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.701413 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.701552 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.702831 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.703254 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.704876 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.707281 23852 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:48.708999 23852 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:48.709875 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.710847 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.711828 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.712945 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.713352 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.508837 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.729256 23926 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:48.735095 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.735111 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.736198 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 241203us
I20260709 12:14:48.737524 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.744644 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.748605 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.749428 27792 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:48.750766 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.751173 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.739113 25114 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.563946 27853 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:48.758201 27932 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:48.762825 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:48.762858 27880 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:48.392880 27936 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:48.773180 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 43517.9us
I20260709 12:14:48.777383 23860 raft_consensus.cc:2376] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7. Re-sending same reply.
I20260709 12:14:48.793123 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.551975 27994 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.392596 27955 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.799772 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.800760 23929 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:48.801004 23852 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:14:48.801882 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 63144.4us
I20260709 12:14:48.802325 27955 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.804196 27631 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:48.805020 27631 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:48.807051 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.414216 27987 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.811986 27792 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:48.812647 27792 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:48.813115 27792 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:48.814081 27877 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:48.816938 23930 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:48.818076 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.806594 27631 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.792649 27932 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:48.826061 23928 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.824515 23933 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:48.419384 27973 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.836184 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.827127 27932 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:48.838328 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:48.842628 23933 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.830606 28000 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:14:48.845995 23934 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
W20260709 12:14:48.846555 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 83991.5us
W20260709 12:14:48.856635 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 358957us
I20260709 12:14:48.857534 27999 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:48.858111 27999 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:48.859155 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:48.859313 27999 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.866974 27925 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:48.867146 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.870476 27950 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.871210 27935 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:48.872030 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52144.7us
I20260709 12:14:48.874368 27853 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:48.861985 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.875713 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:48.879266 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.879474 27991 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:48.880187 25177 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:48.875144 27853 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.880381 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:48.883965 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.884545 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.884717 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.885651 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.885924 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.886647 23930 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:48.887292 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.888094 23930 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:48.888310 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.887884 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.883790 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.887311 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:48.894164 25173 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.887399 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.889587 23933 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:48.889716 23930 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:48.903079 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64753.9us
I20260709 12:14:48.908279 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.910882 27980 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.911742 27991 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:48.912317 23853 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
I20260709 12:14:48.884899 25170 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:48.912647 27994 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.913446 27760 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:48.914055 27760 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:48.915221 27760 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.918126 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.921243 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.922072 23859 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:48.919313 27980 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:48.933207 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 53032.9us
I20260709 12:14:48.936739 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.944445 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.948841 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.950083 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.953338 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:48.954653 27968 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.001s
I20260709 12:14:48.958500 27967 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.960300 27877 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:48.961054 27877 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:48.962859 23852 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:48.962488 27877 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:48.963848 27999 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.966370 27936 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:48.969049 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.970162 23929 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:48.972275 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:48.973073 23934 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.978585 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:48.980791 23853 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:48.981076 27950 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.981838 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.981950 27853 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:48.964013 28005 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:14:48.988986 28005 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:48.989385 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:48.990293 23859 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:48.973024 27974 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.001s
I20260709 12:14:48.992434 27974 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:48.994748 27974 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:48.988058 23854 raft_consensus.cc:2368] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
I20260709 12:14:48.997189 23855 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:49.005067 27973 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.989396 28005 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:49.008627 28005 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.010298 28005 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:49.010429 27792 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.011932 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.012920 27987 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:48.992578 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 133614us
I20260709 12:14:49.016016 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:49.016232 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.017617 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:48.946604 27935 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.019897 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:49.020452 23855 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:49.020252 28000 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:49.022235 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.022505 28000 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:49.022949 28000 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.023766 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.024374 23860 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:48.998162 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:48.955197 27968 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.027643 23925 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:14:49.019955 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.029044 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.031116 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.031706 27967 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:49.028620 27968 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:49.020859 27817 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:49.034039 27817 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:49.034933 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.035321 27817 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.019351 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:49.038409 23852 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:49.032546 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:49.043809 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.032632 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.050765 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.052558 23930 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:49.053673 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.054327 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.055553 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.056679 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.090773 23934 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:49.095965 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.099891 23855 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:49.101732 23749 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.68:43683 timed out after 1.878s (SENT) [suppressed 77 similar messages]
I20260709 12:14:49.102172 27792 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:49.103811 23925 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:49.104028 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.104717 23926 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:14:49.105247 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.104655 23852 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:49.108497 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.125679 23859 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:49.130131 27989 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:49.130785 27989 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.131188 27989 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:49.132057 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 112928us
I20260709 12:14:49.132923 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.133740 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.135824 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.136380 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:49.145905 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:49.151607 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 62321.2us
I20260709 12:14:49.151829 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.098624 25170 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:14:49.166657 23933 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:49.166399 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.164902 23860 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:49.168167 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.170704 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.177569 23932 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:49.180501 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.179608 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.181376 23934 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:49.055438 27931 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:14:49.187376 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.191906 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.198302 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.200598 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.204564 23933 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:49.189308 27967 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:49.177172 27932 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.221865 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.222592 23934 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
W20260709 12:14:49.224159 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 71724.6us
W20260709 12:14:49.225073 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 123384us
I20260709 12:14:49.212157 27631 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:49.229393 23776 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:49.234737 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.237048 23777 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:49.247159 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69402.8us
I20260709 12:14:49.231626 27770 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:49.260638 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 109775us
I20260709 12:14:49.263971 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.263062 27792 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.273303 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.249791 27955 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.276546 23933 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:49.277298 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.279785 28014 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.284809 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.286805 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.249150 27967 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 12:14:49.288446 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.295626 28000 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.296443 23779 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:49.297588 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.298257 23779 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.298765 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.299414 23783 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.301046 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.302120 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.303047 23782 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.298768 28014 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.303766 27935 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:49.305006 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.305282 28014 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:49.302330 27908 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:49.310678 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 47179.2us
I20260709 12:14:49.282861 27991 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 12:14:49.316768 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 99014.3us
I20260709 12:14:49.298329 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.304919 27935 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:49.322104 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.322486 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.329277 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.330649 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.332003 23784 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:49.332994 23931 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:49.333449 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.334385 23782 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:49.334764 28012 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:14:49.334867 23779 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:49.340281 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.293241 27792 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.347760 23780 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:49.350489 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.347254 23779 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:49.351234 23780 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.351697 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:49.332996 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.356600 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:49.333369 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50849.7us
I20260709 12:14:49.352034 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.354199 27932 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:49.354858 27792 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:49.355144 23777 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:49.349301 25168 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:49.360846 23783 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:49.356660 23925 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:49.362001 23781 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.365586 23778 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:49.367290 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.367949 23783 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:14:49.368441 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55530.1us
I20260709 12:14:49.369839 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.369983 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.370813 23778 raft_consensus.cc:2410] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:14:49.370752 23779 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:49.371523 23782 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:49.320416 28014 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:49.331218 27989 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.357928 27935 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.379174 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.377954 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.380676 23776 raft_consensus.cc:2368] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14.
I20260709 12:14:49.377835 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.382485 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.383217 23777 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.333527 27955 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.382678 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.384588 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.387044 23783 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:49.386004 23782 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 6: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.390267 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.392561 28025 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:49.393044 28025 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:49.393990 28025 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.396579 28000 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.400149 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.401083 23784 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:49.402034 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 101107us
I20260709 12:14:49.402493 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.402876 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.404083 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.403735 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.403920 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.408972 23777 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.395390 28025 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.409870 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.384193 27991 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:49.411911 23785 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.394713 27936 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:49.375197 27760 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:14:49.413349 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.418150 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.414716 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.420037 23781 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:49.414745 23778 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.416287 27936 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:49.414535 23783 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.414963 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.441815 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.421561 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.386590 27974 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:14:49.440074 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.407426 27853 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:49.419687 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:49.412539 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 90130.7us
W20260709 12:14:49.469458 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 60973.8us
I20260709 12:14:49.427205 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.413764 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.401950 27935 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.475287 23784 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.476353 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.477258 23784 raft_consensus.cc:2410] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:14:49.478557 23784 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:49.479928 23784 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:49.480855 27880 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.481401 27880 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.482151 23784 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:49.483705 23784 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:49.484172 27880 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.486022 23784 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:49.487331 23784 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:49.488897 23784 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:14:49.489575 27936 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.490929 27936 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.495800 23784 pending_rounds.cc:85] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 6
I20260709 12:14:49.496461 23784 pending_rounds.cc:107] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:14:49.497166 23784 raft_consensus.cc:2889] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:49.495786 28008 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.360s	user 0.002s	sys 0.000s
W20260709 12:14:49.545199 28010 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.334s	user 0.000s	sys 0.004s
I20260709 12:14:49.545661 23776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:14:49.466104 27980 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:14:49.552268 23777 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.553186 27935 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.554291 27955 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.552290 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.558339 25177 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.559453 27784 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:49.560984 27792 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:49.561391 27932 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:49.545234 23785 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:49.575035 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.551383 25173 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:14:49.562536 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 62680.2us
I20260709 12:14:49.562979 23782 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:49.565990 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.576579 23783 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.567144 23776 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:49.567478 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:49.531692 28000 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:14:49.571628 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 158101us
I20260709 12:14:49.577091 23776 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.562175 27989 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.584856 23779 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:14:49.587982 23780 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:49.572813 27968 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.569865 27631 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.589061 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:49.593979 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:49.595140 23779 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:49.596118 27908 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:49.596324 28030 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:14:49.570968 27853 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:14:49.598830 27853 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:49.599185 27853 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.600134 27853 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.601958 27853 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:49.578219 28000 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:49.613390 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:49.618963 23780 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:14:49.622439 23900 leader_election.cc:400] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
I20260709 12:14:49.622816 23900 leader_election.cc:403] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:49.629204 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.629122 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.632089 23926 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.633592 23778 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:49.636473 23780 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:49.637319 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:49.580895 27880 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:14:49.640837 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.641803 27955 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:49.642874 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:49.648335 23778 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:14:49.651427 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:49.656002 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.651491 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.661764 28033 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:49.648469 27908 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.619969 27980 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:14:49.666527 27581 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9.
I20260709 12:14:49.669916 27877 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:49.666546 27770 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:49.671926 27967 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:49.671523 28000 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:49.674091 23782 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:49.647109 27968 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:49.671092 28031 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:14:49.703248 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:49.666492 27908 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:49.710433 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 124248us
I20260709 12:14:49.710980 23926 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:49.686743 27877 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.711861 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:49.695616 27770 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:49.714726 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 124867us
I20260709 12:14:49.693660 28033 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.717954 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:49.720949 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:49.700570 27967 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.717911 27931 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:49.741858 23900 leader_election.cc:400] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
I20260709 12:14:49.742338 23900 leader_election.cc:403] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:14:49.710784 27760 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:14:49.744670 28032 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
W20260709 12:14:49.753377 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.755447 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.718670 27877 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.720075 27770 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.764808 23851 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:14:49.759222 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45635.2us
I20260709 12:14:49.765746 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.710443 27989 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:14:49.772123 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.766633 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.764127 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.758689 23776 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.777072 23777 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 11. Ops: []
I20260709 12:14:49.757848 28000 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:49.773850 27931 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:49.751792 27925 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:14:49.789134 23776 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:49.794816 23783 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:49.791484 27931 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.796970 27931 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.798552 23857 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:49.799368 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 126484us
W20260709 12:14:49.800654 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.546s (SENT)
W20260709 12:14:49.807502 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 59973us
I20260709 12:14:49.808363 23855 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:49.810175 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:49.811394 23851 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:49.814157 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.814834 23925 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:49.766278 27974 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:14:49.816972 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.824235 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.791738 27792 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:49.827809 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.867823 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:49.828778 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 76511.6us
I20260709 12:14:49.797066 28035 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:49.830770 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:49.831415 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:49.832058 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.833168 27963 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:49.838277 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:49.839197 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:49.841521 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:49.843458 27953 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:49.916215 23925 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:49.844952 27949 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:49.845739 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:49.846451 28040 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:49.847225 27784 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:49.847514 23857 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:49.848896 27631 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:49.858457 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.859261 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:49.922829 23934 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:49.860096 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.861424 25170 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:49.862421 25172 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:14:49.864581 25176 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:49.865622 27760 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:49.865963 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.868562 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.894768 23931 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.824550 27999 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:49.924098 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 78432us
I20260709 12:14:49.918768 23858 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:49.932782 27999 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:49.910589 27963 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:49.780345 28033 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.936812 27999 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:49.940466 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 86843.4us
W20260709 12:14:49.943315 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 185419us
I20260709 12:14:49.944887 25173 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
W20260709 12:14:49.948807 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 118562us
I20260709 12:14:49.949641 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:49.954491 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116351us
I20260709 12:14:49.954818 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.891916 27967 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.946393 27955 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:49.978582 28036 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:49.979058 28036 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:49.979929 28036 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:49.981190 28036 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.984251 23858 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:49.988461 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:49.994503 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:49.954416 27935 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:49.956071 27932 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:49.996783 23931 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:14:49.998842 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49625.5us
I20260709 12:14:49.999509 23926 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:49.999909 23856 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:50.009572 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:49.970854 27950 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:49.995814 27792 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.021292 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:49.997041 28035 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.022192 27949 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:49.997464 28042 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:50.011217 27968 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.052927 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:50.054123 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.018033 28005 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:50.056078 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.023815 27973 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:50.060153 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 84527.3us
I20260709 12:14:50.061087 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.054569 27923 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:50.062052 27923 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.056384 27953 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:14:50.063431 28045 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.052234 27974 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:14:50.054837 27994 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:14:50.065222 27994 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:50.064062 28045 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.064949 28005 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.066907 28044 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:50.067384 28044 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:50.067862 28044 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.066339 28032 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:50.068595 28032 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.068606 28005 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.069568 28032 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.070960 28032 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.071120 28045 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.064881 27973 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:50.072746 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44820.3us
I20260709 12:14:50.074292 23825 leader_election.cc:400] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14.
I20260709 12:14:50.074340 27973 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.062994 27923 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.077136 27923 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.075367 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.080348 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:50.081238 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88481.9us
W20260709 12:14:50.083477 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 85237.2us
I20260709 12:14:50.090701 25172 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:14:50.068156 28038 log.cc:927] Time spent T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:14:50.123291 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.056492 27967 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.123196 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.127842 28041 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.053977 28042 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:50.116319 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:50.132583 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.130563 25170 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:50.065663 27994 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.136101 25172 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.121352 27932 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:50.141409 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 94206.3us
I20260709 12:14:50.142540 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.144421 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:50.134902 27853 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.003s
W20260709 12:14:50.146745 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 85760.5us
I20260709 12:14:50.063113 28035 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.134779 27935 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.149468 25177 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:50.133843 27955 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.139963 27925 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.153398 28052 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:50.154459 27994 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.180298 23851 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:50.156810 28049 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.157508 28059 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.158712 28044 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.196297 28044 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.148447 28050 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:14:50.138518 28042 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:50.165536 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.170833 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 95570.6us
I20260709 12:14:50.204270 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.144351 27932 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.150437 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:50.148758 27908 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:50.138720 28041 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:50.207211 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.205154 27932 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.213021 27853 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:50.227177 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.237246 28042 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.239662 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.242916 28055 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:14:50.243904 28041 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.244511 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.245172 28059 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.247231 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.248212 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:50.247571 28043 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:14:50.254055 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:50.255409 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:50.257498 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 91123.3us
I20260709 12:14:50.258387 23776 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:50.153267 27792 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.280501 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.261205 23932 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:50.247478 27931 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:50.302783 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 58383.9us
I20260709 12:14:50.280558 28033 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:50.248378 27980 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:50.302871 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.301724 28053 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:50.247516 28059 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.281008 27936 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.278857 27991 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:50.316030 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.258862 28052 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:50.257275 27994 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:50.308023 28012 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:50.282351 28055 consensus_queue.cc:1243] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:50.310840 27908 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:50.320387 27784 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:50.303388 28044 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:50.325306 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:50.326633 28051 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 12:14:50.326929 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.314828 27980 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:50.333199 28000 consensus_meta.cc:287] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.000s
I20260709 12:14:50.322484 27955 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.322630 27991 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.319705 28005 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 12:14:50.336869 27770 consensus_meta.cc:287] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.000s
I20260709 12:14:50.312867 28033 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:50.321781 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.325848 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:50.364236 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:50.327761 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.317274 23680 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:50.354727 27931 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.356690 28042 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.341460 27853 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.346735 27980 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:50.357615 27999 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:50.372985 27877 consensus_meta.cc:287] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.001s
I20260709 12:14:50.259927 28045 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:14:50.372785 28036 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:50.342729 28044 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:50.392590 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 70612.3us
W20260709 12:14:50.392800 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65116.5us
I20260709 12:14:50.402804 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:50.376955 28055 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.386595 28042 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:50.404996 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.406603 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:50.407938 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:50.407989 28039 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:50.408488 28039 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:50.409260 28039 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.410467 28039 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.416947 27991 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.420574 23857 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.370041 27770 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:50.424255 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 102516us
I20260709 12:14:50.425004 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.429889 25168 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:14:50.430909 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.400147 28036 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.393954 27974 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.396166 27987 raft_consensus.cc:2764] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:14:50.420636 27967 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.426936 27880 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.470687 23860 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:50.457316 28036 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:50.475622 25169 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:14:50.472216 25172 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
W20260709 12:14:50.472800 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64953.2us
I20260709 12:14:50.390026 28033 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.392328 27877 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.476564 23931 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:50.472538 27784 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:50.492882 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:50.509907 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.478266 27967 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:50.516077 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 114788us
I20260709 12:14:50.510465 23779 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:50.467491 27974 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.518536 23825 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.519380 27968 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.521580 27968 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.524065 28032 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:50.524725 27932 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.526938 27932 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.527657 23777 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:50.529047 27770 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.530828 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:50.531746 28070 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:14:50.532505 28075 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.479308 27987 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:14:50.534333 27987 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:50.534735 27987 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.535638 27987 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.540969 27987 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:50.542755 28040 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:50.542073 28000 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Committing config change with OpId 8.7: config changed from index 4 to 7, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:50.543185 28040 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.544173 28040 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.524750 28032 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.545404 28040 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.546221 28032 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.541900 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:50.548084 23854 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:14:50.552009 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.551436 23782 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 4 to 7, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:50.556421 27968 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.558293 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.560781 27877 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:50.532263 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55202.6us
I20260709 12:14:50.563196 27963 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.564617 27963 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.567423 28049 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:50.567842 28049 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.568692 28049 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.569852 28049 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.573161 27974 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.573791 27974 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:50.575034 27923 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:50.575433 27923 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:50.576192 28065 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.133s	user 0.000s	sys 0.003s
I20260709 12:14:50.576440 27923 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:14:50.575433 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 58588.4us
I20260709 12:14:50.577760 27923 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.578357 27770 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:50.578608 28033 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.579241 27770 consensus_queue.cc:1243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:50.579703 27631 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:50.580091 27631 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:50.580931 27631 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.582052 27631 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:50.580945 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.584395 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.585139 27880 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.563421 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.586843 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.589820 27991 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:50.590070 27999 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.592186 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.607734 23852 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:50.593406 27968 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:14:50.608345 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 115532us
I20260709 12:14:50.613041 23857 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:14:50.608572 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 118055us
W20260709 12:14:50.609480 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46141.5us
W20260709 12:14:50.619360 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.616916 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.611342 27967 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:50.621928 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:50.622620 28039 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.622846 27908 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:50.624313 23934 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:50.624934 28050 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:50.617877 28005 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.626793 23858 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:50.621426 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:50.641076 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:50.641960 27935 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:50.642328 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50222.8us
I20260709 12:14:50.618415 28070 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.643523 27980 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.644083 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.645315 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.652206 27968 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:50.652699 27968 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.653621 27968 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.654876 27968 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:50.668195 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 75755.5us
I20260709 12:14:50.620955 28000 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:54770 (request call id 0) took 1055 ms. Trace:
I20260709 12:14:50.680259 23680 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:50.660952 28050 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:50.661142 28040 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:50.679112 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:50.692323 27935 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:14:50.692677 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 75853.3us
I20260709 12:14:50.693432 27581 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:50.695811 27950 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:50.696219 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.701048 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55817.2us
I20260709 12:14:50.708140 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.680344 28000 rpcz_store.cc:276] 0709 12:14:49.553447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.555808 (+  2361us) service_pool.cc:224] Handling call
0709 12:14:49.591585 (+ 35777us) log.cc:844] Serialized 494 byte log entry
0709 12:14:50.607226 (+1015641us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3945,"raft.queue_time_us":366,"raft.run_cpu_time_us":1917,"raft.run_wall_time_us":82266,"thread_start_us":38090,"threads_started":1,"wal-append.queue_time_us":64448}
I20260709 12:14:50.710417 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.711977 28040 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.712206 27935 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.712649 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:50.713912 27935 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:50.714372 28050 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:50.714373 27935 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.715436 27935 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:50.677974 27994 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:50.716046 27994 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.667634 28005 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.718564 27994 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.675897 28070 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:50.719178 23784 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:50.719866 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e60a1c6520545729d27474d8f237ce5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:14:50.655997 28039 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.707267 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.704278 27581 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.713984 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:50.723150 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54262.2us
I20260709 12:14:50.733949 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.731096 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:50.702093 27980 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.738566 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:50.737903 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:50.746234 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.705111 27967 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:14:50.710875 28036 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.744444 27908 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:50.763903 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56739.3us
I20260709 12:14:50.750743 28050 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:50.756381 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.719082 28005 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.730427 27770 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 12:14:50.761823 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50242.1us
I20260709 12:14:50.768491 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.769440 25168 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:50.750254 28032 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:50.752059 27980 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.772768 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.776304 23857 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:50.756834 27932 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.779042 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.763657 27908 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:50.755080 28081 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:50.791863 28070 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.794358 28036 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.781925 27581 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.798326 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:50.808954 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.782428 27980 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.803630 27908 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:50.813825 28084 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:50.782322 28041 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.781893 27950 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:50.836118 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 57178.8us
I20260709 12:14:50.836984 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.837787 27980 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:50.803769 27932 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:50.840263 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:50.840904 23931 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:50.840790 27932 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.843887 28075 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.827849 28049 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:50.845476 28039 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.809626 27994 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:14:50.848541 27581 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.849958 28040 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:50.786201 27974 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:14:50.804080 27987 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:14:50.856225 27989 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:50.856752 27989 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:50.857659 27989 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:50.860101 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.803725 28081 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.865597 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:50.875232 23931 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:50.875790 28081 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:50.864953 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:50.884631 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.887775 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 106170us
I20260709 12:14:50.888957 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.878669 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:50.893316 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:50.858946 27989 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:50.893807 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 56914.3us
I20260709 12:14:50.895471 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.904136 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:50.893378 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:50.913364 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 103222us
I20260709 12:14:50.891948 28075 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:50.877712 28044 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.894654 28039 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:50.906524 25169 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:50.901286 28032 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.903496 27968 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:14:50.928122 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.911109 28041 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:50.910351 28049 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:14:50.944005 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65864.3us
I20260709 12:14:50.913009 27880 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:50.919062 28053 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:50.927158 28039 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.926801 28093 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.941004 27784 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:50.849519 28045 consensus_queue.cc:1175] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "438159d8de3c4027bdcf57eb81dedf86" responder_term: 12 status { last_received { term: 11 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:50.948104 27880 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:50.969504 28078 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.250s	user 0.003s	sys 0.000s
W20260709 12:14:50.969769 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.970145 28092 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:50.983248 28092 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.983847 27925 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.984369 27925 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.985446 28056 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:50.986157 28092 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.985971 28056 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:50.986478 27925 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.987804 27880 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.018317 23851 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:51.018157 23680 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:51.019486 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:51.020429 23858 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:51.020057 23680 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 11, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 11 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:50.975396 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:50.975932 27631 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:50.976686 28032 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:50.979931 28093 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:50.980770 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:50.956612 28053 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:50.988874 27784 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:50.991938 28077 log.cc:927] Time spent T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.286s	user 0.003s	sys 0.000s
I20260709 12:14:50.992794 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.993536 23933 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:50.994257 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:50.965039 28039 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:50.995645 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:50.972853 28075 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.001721 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:51.063623 25168 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:51.002251 28056 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:50.997870 23929 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:51.004680 27963 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:51.066749 27963 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:51.067144 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.067757 27963 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.067792 23929 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.069265 27963 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.069913 23851 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:51.008601 28079 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.260s	user 0.000s	sys 0.002s
I20260709 12:14:51.071106 23859 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:51.009573 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:51.047811 28092 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.072026 23925 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:51.072556 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.073189 23857 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:50.972455 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.013643 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.074539 23854 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 7: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:51.022598 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:51.075390 23934 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:14:51.021513 23680 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:51.029912 28052 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:14:51.023032 27980 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.077571 27980 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:51.048990 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 68311.1us
I20260709 12:14:51.080798 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:51.054539 23858 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:51.056236 23928 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:51.055263 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.061856 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.096647 23932 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:51.044374 28053 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:51.101890 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:51.105168 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:51.063380 27631 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:51.032615 27935 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.112010 27955 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.111811 27963 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:51.090498 27792 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:51.110754 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:51.116503 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.126372 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:51.062335 25171 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:51.085601 28092 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.111718 23680 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 12, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 12 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:51.094720 28000 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.108093 28052 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:51.103957 27999 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:51.141844 28040 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:51.094622 27991 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:14:51.143004 27935 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:51.116824 28005 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:14:51.120039 27974 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:14:51.145061 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.148394 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.213604 23927 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:51.146220 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:51.214375 23932 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:14:51.120980 28084 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:51.145669 27963 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:14:51.152886 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.154846 23929 raft_consensus.cc:2410] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:51.127859 27631 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.125499 27908 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.152668 28040 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:51.152900 28000 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.154906 27999 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.142164 23928 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:51.125432 27980 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:51.209013 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 236601us
I20260709 12:14:51.155663 23855 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:51.220088 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 93777.4us
I20260709 12:14:51.155081 28052 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:51.200920 27877 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.216460 28084 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.227577 27999 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.239410 28000 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:51.240087 23680 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:51.245935 27784 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:51.246752 28052 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.251472 27980 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.252310 27770 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.253705 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.254470 23927 raft_consensus.cc:2410] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:51.266784 27989 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:51.269192 27631 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:51.269811 23857 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:51.328245 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 195261us
I20260709 12:14:51.272148 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.272452 28092 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:51.273080 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.273634 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:51.278649 28039 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.220290 27963 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:51.279556 28100 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:14:51.226068 28040 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.281612 27880 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.283360 28081 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.208125 28093 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.294989 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:51.340691 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 45800.5us
I20260709 12:14:51.307408 23933 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:51.312937 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 81694.9us
I20260709 12:14:51.270952 27908 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:51.312562 28084 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.329269 23852 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:51.331480 27877 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.345288 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:51.338208 25171 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:51.340198 28053 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.351969 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.375998 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.352977 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.356194 23777 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:51.352648 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.353417 23781 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:51.364674 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:51.372968 23852 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:51.381235 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.363317 23930 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:51.352927 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.382578 23780 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:51.379078 23926 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.381825 23781 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.359112 28039 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:51.387777 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 53010.4us
I20260709 12:14:51.389187 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.349754 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.389050 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:51.394220 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:51.327245 27980 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.401774 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:51.402729 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.403841 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.404404 23776 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:51.405278 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.405974 23776 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:51.397626 23778 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:14:51.399821 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.407792 23928 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.329654 27770 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.414629 27770 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:51.400640 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:51.401271 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.415752 27908 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:51.410535 28110 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.394588 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.385198 28045 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:51.347875 28075 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:51.417953 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.417893 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:51.426417 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.419212 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.438315 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:51.421460 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:51.454247 25172 raft_consensus.cc:2410] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:51.425818 23925 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:51.361591 27880 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:51.452801 23854 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:51.428018 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.363124 28081 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:51.460757 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.468545 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:51.450160 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.470803 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:51.482786 23776 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.426882 23777 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:51.426373 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.484329 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.484980 23929 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.493312 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:51.510113 23781 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:51.528978 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:14:51.529819 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64562.8us
I20260709 12:14:51.461849 28075 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.562925 23926 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:51.563225 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.416523 27908 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.563913 23781 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.564623 23928 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:51.531922 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.468103 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:51.532928 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:51.501222 25169 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:51.426342 27581 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:51.450922 27936 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.485488 28112 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:51.534759 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.576083 23777 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:14:51.576659 23928 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:51.533970 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.534350 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:51.369240 27877 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.562390 23780 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.483117 23932 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:51.578078 23779 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:51.494778 27991 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:51.566294 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 98277us
I20260709 12:14:51.480768 28045 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.399287 28032 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:14:51.586766 23783 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:51.589882 23783 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:14:51.572670 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:51.540503 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:51.354650 27963 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.486217 27955 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 12:14:51.567178 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:51.533012 28109 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.149s
I20260709 12:14:51.486712 28044 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:51.573894 23926 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:51.570812 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:51.574491 23934 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:51.541666 23925 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:14:51.584187 23825 leader_election.cc:336] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.513s (SENT)
I20260709 12:14:51.584789 23776 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:51.612689 23785 raft_consensus.cc:2376] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11. Re-sending same reply.
I20260709 12:14:51.801532 23777 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:51.631404 23680 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:51.664503 23859 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:14:51.575868 28039 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.680855 23900 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:51.684893 23852 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:51.665072 23780 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:51.635941 23929 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:51.612860 23927 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:51.645911 25174 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:14:51.773037 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 241188us
I20260709 12:14:51.818650 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.593823 23779 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:51.569996 28075 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:51.821467 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 237289us
W20260709 12:14:51.630393 28101 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
W20260709 12:14:51.783900 28113 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:14:51.584841 28081 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.584868 27991 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:51.777989 28111 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 12:14:51.827818 23825 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.618s (SENT)
I20260709 12:14:51.694895 23854 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:51.586787 28045 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.698904 27581 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.830309 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.857774 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.859460 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:51.847747 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 167000us
I20260709 12:14:51.628741 28096 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:51.802495 28053 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:51.806154 28080 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:51.819789 27980 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:51.843498 28049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.001s	sys 0.010s
I20260709 12:14:51.583135 27936 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.881363 23928 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:51.580570 27932 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.256s
I20260709 12:14:51.595950 28112 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.854743 28032 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.630501 28052 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.863305 27923 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:51.878052 28049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.001s	sys 0.012s
I20260709 12:14:51.652879 28084 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:51.868523 28118 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:51.865114 28096 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:51.866495 28093 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:51.959839 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 132043us
I20260709 12:14:51.883870 27991 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.879774 27974 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:51.882903 27908 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:51.876780 27925 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:14:51.980173 28096 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.981487 28096 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.985937 27923 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:51.986465 27923 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:51.989173 28112 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:51.991907 28123 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:51.992584 28123 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:51.993552 28123 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:51.996196 28115 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 12:14:51.996368 23825 leader_election.cc:336] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.647s (SENT)
W20260709 12:14:51.997730 23825 leader_election.cc:336] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.673s (SENT)
I20260709 12:14:51.998189 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:51.996114 27932 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:51.856416 28081 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.999346 27932 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:51.868433 27581 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:51.999759 28040 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
W20260709 12:14:51.971742 28105 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.704s	user 0.005s	sys 0.003s
W20260709 12:14:52.022724 28101 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:14:52.005774 27908 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:52.042934 27936 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:51.987347 27923 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:52.126200 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.052s
W20260709 12:14:52.127306 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.052s
W20260709 12:14:52.135419 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.136627 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 567) took 1146 ms. Trace:
I20260709 12:14:52.027951 28040 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.137081 23931 rpcz_store.cc:276] 0709 12:14:50.981744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.986659 (+  4915us) service_pool.cc:224] Handling call
0709 12:14:50.987023 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.987379 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.987406 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:51.010917 (+ 23511us) log.cc:844] Serialized 26 byte log entry
0709 12:14:51.107752 (+ 96835us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:51.111789 (+  4037us) log.cc:844] Serialized 14 byte log entry
0709 12:14:51.119216 (+  7427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.119292 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:51.629803 (+510511us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:52.112018 (+482215us) raft_consensus.cc:1685] finished
0709 12:14:52.128443 (+ 16425us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:52.128595 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25577,"prepare.queue_time_us":4103,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":13369,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":4097,"spinlock_wait_cycles":73726848,"thread_start_us":96456,"threads_started":1,"wal-append.queue_time_us":189950}
I20260709 12:14:52.146277 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 580) took 1124 ms. Trace:
I20260709 12:14:52.089262 28049 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:52.109681 27877 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.146908 23933 rpcz_store.cc:276] 0709 12:14:51.012405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.061572 (+ 49167us) service_pool.cc:224] Handling call
0709 12:14:52.124798 (+1063226us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54000e3154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:52.136908 (+ 12110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268863744}
I20260709 12:14:52.156133 27963 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.157449 27936 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.161638 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.021s
W20260709 12:14:52.168112 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.000s	sys 0.021s
I20260709 12:14:52.165216 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:52.175921 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 605151us
I20260709 12:14:52.148478 27923 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:52.174489 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.182235 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:52.184333 23784 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:52.185495 27991 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.185830 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:52.160691 27908 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.186570 27908 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.187762 27991 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.184382 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.189919 23776 raft_consensus.cc:2410] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:52.185751 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.195822 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:52.196475 23781 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:52.190017 27999 consensus_meta.cc:287] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.861s	user 0.013s	sys 0.000s
I20260709 12:14:52.161087 28049 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.228480 23781 raft_consensus.cc:2410] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:52.176395 27936 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.240132 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.218564 28081 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.218367 27955 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:52.250285 23781 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:52.247298 23778 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:52.252827 27936 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.253459 28044 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.253983 28044 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.254575 28050 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:52.254966 28050 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.255872 28050 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.257107 28050 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.260666 28045 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:52.261937 28103 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 1.009s	user 0.010s	sys 0.009s
I20260709 12:14:52.262976 28045 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.218703 28084 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.263828 27987 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:52.244997 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.249553 28039 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:52.264992 28044 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.200424 28112 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:14:52.229691 27908 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.236323 28123 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
W20260709 12:14:52.267575 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 69988.9us
I20260709 12:14:52.271230 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.283181 23776 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:52.224927 27999 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.272799 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:52.273490 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.239509 27932 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:52.275547 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.279619 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:52.236908 25171 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:52.233011 27877 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:14:52.287557 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.239012 28122 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:14:52.289510 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.302376 23780 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:52.285643 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.304601 23778 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:52.268855 27581 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.278345 28084 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.282500 27949 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.300122 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.270844 27987 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:52.318760 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:52.319538 23854 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.290620 28044 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:52.323004 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:14:52.296550 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51633us
I20260709 12:14:52.324599 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.282235 27968 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.317641 23852 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.273008 27955 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.292032 28109 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:52.312958 28075 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.326977 27908 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:52.331166 28052 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:52.329713 25113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.233s	user 0.011s	sys 0.015s
I20260709 12:14:52.322965 28084 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.320458 27877 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:52.329739 27987 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.334555 27987 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:52.329833 27968 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.337670 27968 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.330420 27955 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:52.339149 28114 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.721s	user 0.000s	sys 0.007s
I20260709 12:14:52.330821 28109 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:52.333365 25113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.237s	user 0.011s	sys 0.015s
W20260709 12:14:52.340926 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.24481e+06us
I20260709 12:14:52.340490 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 601) took 1010 ms. Trace:
I20260709 12:14:52.309173 28039 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.353299 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:52.356703 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:52.314134 28126 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.355741 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60350.9us
I20260709 12:14:52.359545 23860 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.353410 23859 rpcz_store.cc:276] 0709 12:14:51.330299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.330420 (+   121us) service_pool.cc:224] Handling call
0709 12:14:51.662759 (+332339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:51.804747 (+141988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.340270 (+535523us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54002d46d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:52.340419 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747044608}
I20260709 12:14:52.363334 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:52.317070 28131 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:52.322641 28040 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.341581 27991 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:52.354138 28130 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.368186 25174 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:52.369494 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.381712 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.382831 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.372198 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:52.367419 27908 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:14:52.372942 28112 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:52.364794 28044 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:52.366374 25168 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:14:52.368494 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:52.340490 28109 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.363437 28039 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.357513 28093 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.357067 28075 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.390223 23860 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:52.370888 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.393863 28040 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.326973 28124 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:14:52.323385 27949 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.408748 25169 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:14:52.416067 28128 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:14:52.358175 28121 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:14:52.420480 28133 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:52.374745 27974 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:14:52.368281 28126 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.325784 28110 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:52.400609 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:52.478353 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 77839.6us
W20260709 12:14:52.430812 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68614.2us
I20260709 12:14:52.373395 27936 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:14:52.428512 27908 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.509106 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.435374 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:52.394078 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:52.439738 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68918.9us
I20260709 12:14:52.509060 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:52.511089 23927 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.439491 28109 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.404277 28084 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.440291 28137 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:52.391921 28130 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:52.402887 27877 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:14:52.441228 27932 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:52.553887 27932 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.443032 28040 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.555091 27932 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.556630 28040 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.557816 23776 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:52.403887 28122 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:14:52.397320 28112 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:52.448887 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:52.559085 28130 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:52.559480 25169 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:52.559048 28112 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:52.560657 27987 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:52.561177 27987 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.518285 28137 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.449589 28032 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:52.456357 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:52.458835 28134 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:52.459896 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:52.574548 25174 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.460253 23854 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:52.461114 28138 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:52.463203 27949 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.437464 28093 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.464922 28049 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:52.465365 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:52.467414 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:52.471400 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.588183 23933 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:52.471721 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.473140 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.430831 27581 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.476785 23680 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 9, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 9 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:14:52.486595 23779 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:52.396860 28123 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:52.477664 27963 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.470762 28053 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:52.516772 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.598557 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.480162 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:52.526608 25171 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:52.526472 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.535986 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:52.537474 28098 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 1.290s	user 0.004s	sys 0.005s
I20260709 12:14:52.542637 28044 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.546232 28126 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.511322 28135 consensus_queue.cc:1059] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:14:52.548090 28039 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.510083 28050 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:52.514716 27908 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.551385 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.552011 27955 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.564185 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 85359.9us
I20260709 12:14:52.584936 28040 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.556814 27932 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.673357 23857 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:52.574283 25176 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:52.596745 27923 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:52.675118 27923 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.570938 28112 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.676698 28135 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.677311 27923 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.677510 25168 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.678349 23854 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:52.599661 27963 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.672439 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.607807 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.673504 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:52.671139 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2653) took 1254 ms (client timeout 1513 ms). Trace:
I20260709 12:14:52.693176 25174 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:52.481120 28110 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:52.587656 23932 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:52.586246 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 162577us
I20260709 12:14:52.598125 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.698746 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.699894 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.700734 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.600032 23784 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:52.669418 28032 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:52.601130 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 571) took 1618 ms. Trace:
I20260709 12:14:52.601294 23778 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
W20260709 12:14:52.602285 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 122206us
I20260709 12:14:52.585583 27877 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.587018 28122 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.437922 28118 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:52.606163 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 214674us
I20260709 12:14:52.564482 28137 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:52.563560 27987 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.585807 27994 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.596956 28123 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:52.670724 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.716934 28140 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:52.717533 28140 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.719748 28123 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:52.722119 28093 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.723831 27994 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.726027 27994 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.702152 28084 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:52.726792 27936 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:52.730715 23860 raft_consensus.cc:2410] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:14:52.697180 28040 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.733049 28040 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.677243 28135 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:52.682703 27955 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.733247 28109 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:52.734128 28109 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:52.734462 28109 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.737262 23780 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:52.735257 28109 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.740779 28109 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.713419 28032 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:52.744163 28032 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.746068 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.605690 27853 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:52.746960 23776 raft_consensus.cc:2393] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:52.604242 28053 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.748143 28053 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.749248 27853 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:52.695065 23785 rpcz_store.cc:269] 0709 12:14:51.361918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.379999 (+ 18081us) service_pool.cc:224] Handling call
0709 12:14:52.616060 (+1236061us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540010ac54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:52.616177 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444493056}
I20260709 12:14:52.750672 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:52.597410 23934 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:14:52.513429 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 130664us
I20260709 12:14:52.715551 27877 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.597342 23927 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:52.700111 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.699386 23859 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:52.701505 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:52.752871 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52832.5us
I20260709 12:14:52.697369 25168 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:52.706488 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:52.705758 23930 rpcz_store.cc:276] 0709 12:14:50.983016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.992487 (+  9471us) service_pool.cc:224] Handling call
0709 12:14:50.992773 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.384677 (+391904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.600891 (+1216214us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54003c28d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:52.601070 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851703680}
I20260709 12:14:52.616607 23783 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:52.694432 28050 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:52.614647 27949 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:14:52.768409 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.711062 23898 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.599542 27581 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.743194 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2695) took 1382 ms. Trace:
I20260709 12:14:52.749660 28053 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.750197 23680 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:52.751274 23683 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 277) took 22608 ms (client timeout 15000 ms). Trace:
I20260709 12:14:52.753321 23854 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.681708 27963 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.781132 28110 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.753947 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.748749 23932 raft_consensus.cc:2410] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:14:52.754865 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48461.7us
I20260709 12:14:52.808596 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:52.696954 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 89199.5us
I20260709 12:14:52.758810 28050 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.716070 28122 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.759749 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.761004 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.816468 23784 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.761860 28084 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:52.763264 27999 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.765249 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.841548 23780 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:14:52.766310 28127 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:52.851374 23932 raft_consensus.cc:1240] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:52.773993 27853 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.853096 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.769457 28045 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:52.757687 28140 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.772557 23777 rpcz_store.cc:276] 0709 12:14:51.360310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.576812 (+216502us) service_pool.cc:224] Handling call
0709 12:14:51.801368 (+224556us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.801921 (+   553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.742955 (+941034us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400370fd0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:52.743105 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515720448}
I20260709 12:14:52.786019 28096 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:52.771224 28044 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:52.713713 28145 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:52.787195 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:52.677650 28112 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.807435 23776 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.856769 28039 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:52.813191 27963 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:14:52.817317 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 105197us
I20260709 12:14:52.717548 27908 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:52.816367 23778 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.854853 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.927608 28125 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:52.856035 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.854264 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.928558 28084 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.858752 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:52.928706 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74535.1us
I20260709 12:14:52.929814 23925 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:52.851828 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:52.930770 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 79040.7us
I20260709 12:14:52.857632 23781 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:52.872018 28040 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:52.770169 27980 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:52.932265 28040 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.860603 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.933440 23784 raft_consensus.cc:2410] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:14:52.781203 23683 rpcz_store.cc:269] 0709 12:14:30.148807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.159999 (+ 11192us) service_pool.cc:224] Handling call
0709 12:14:52.751048 (+22591049us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:48.182166 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:48.233431 (+ 51265us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:48.233474 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:14:48.233490 (+    16us) tablet.cc:662] Decoding operations
0709 12:14:48.247551 (+ 14061us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:48.247647 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:48.247681 (+    34us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:14:48.248509 (+   828us) tablet.cc:701] Row locks acquired
0709 12:14:48.248529 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:14:48.248678 (+   149us) write_op.cc:270] Start()
0709 12:14:48.248789 (+   111us) write_op.cc:276] Timestamp: P: 1783599288248628 usec, L: 0
0709 12:14:48.248814 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:14:48.249508 (+   694us) log.cc:844] Serialized 3676 byte log entry
0709 12:14:48.535934 (+286426us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2aa6a80ac 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b
0709 12:14:48.811120 (+275186us) op_driver.cc:464] REPLICATION: finished
0709 12:14:48.869528 (+ 58408us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b1800190b40. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2ad72f190 00007fb2ad728ae7 00007fb2ad727dd9
0709 12:14:49.034574 (+165046us) write_op.cc:301] APPLY: starting
0709 12:14:49.034723 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:49.705014 (+670291us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:49.705058 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:49.856263 (+151205us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:49.856610 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:14:49.856682 (+    72us) write_op.cc:312] APPLY: finished
0709 12:14:49.857177 (+   495us) log.cc:844] Serialized 89 byte log entry
0709 12:14:49.857412 (+   235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:49.857747 (+   335us) write_op.cc:454] Released schema lock
0709 12:14:49.857816 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:49.935886 (+ 78070us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:49.937294 (+  1408us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34179,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":223007,"apply.run_cpu_time_us":23894,"apply.run_wall_time_us":884927,"cfile_cache_hit":34,"cfile_cache_hit_bytes":35547,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":1,"lbm_read_time_us":127921,"lbm_reads_1-10_ms":2,"lbm_reads_10-100_ms":1,"lbm_reads_gt_100_ms":1,"lbm_reads_lt_1ms":4,"num_ops":9,"prepare.queue_time_us":214311,"prepare.run_cpu_time_us":27272,"prepare.run_wall_time_us":448193,"raft.queue_time_us":938,"raft.run_cpu_time_us":24335,"raft.run_wall_time_us":157859,"replication_time_us":561905,"spinlock_wait_cycles":91040896,"thread_start_us":724865,"threads_started":4,"wal-append.queue_time_us":326875,"wal-append.run_cpu_time_us":7504,"wal-append.run_wall_time_us":2281810}]]}
W20260709 12:14:52.859253 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50776.1us
I20260709 12:14:52.928802 23858 raft_consensus.cc:2410] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:14:52.935982 23825 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:52.856462 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.936882 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:52.868166 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.828563 23927 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:52.873915 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:52.874573 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.938374 23926 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:52.807643 23779 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:14:52.910539 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.872320 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.886996 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:52.849414 28056 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:52.850804 28081 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:52.850639 27991 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.852622 28138 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.921904 28123 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.922693 28075 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:52.876019 28145 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:52.925186 28142 raft_consensus.cc:2764] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:14:52.872398 28139 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:52.927907 23857 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:52.929070 27963 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:52.932554 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:52.908123 27949 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:14:52.818681 28110 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.937559 23930 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:52.932008 23900 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:52.911957 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 124869us
I20260709 12:14:52.941327 23933 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:52.948734 27955 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:52.943894 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.946220 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:52.979875 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.997197 23926 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:52.868480 28096 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:52.875867 28130 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:52.950476 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.011469 23927 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:52.867411 28118 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.925130 28044 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:53.016738 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:52.958004 27880 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:52.871843 28121 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:52.932796 27980 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:52.973884 27963 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:52.910382 27581 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:52.956394 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2672) took 1547 ms. Trace:
I20260709 12:14:52.941141 28005 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:52.946169 28056 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.947022 27908 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:52.945935 28081 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.872937 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:52.861209 28127 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.072285 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:53.072988 25169 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:52.867614 28045 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:52.942245 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:53.020167 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:52.867823 27974 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:14:52.980544 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:52.980854 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:53.077234 23852 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:53.002938 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70998us
I20260709 12:14:53.077425 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:53.078356 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:52.981138 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:53.014104 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.079113 23855 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.079196 23926 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:52.951231 27991 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.954133 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.027958 23933 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:14:52.956313 28075 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.047855 23783 rpcz_store.cc:276] 0709 12:14:51.409347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.591228 (+181881us) service_pool.cc:224] Handling call
0709 12:14:52.616204 (+1024976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:52.753622 (+137418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:52.956192 (+202570us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540010ac50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:52.956337 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354867968}
I20260709 12:14:52.973701 28139 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.012696 28131 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:52.980224 28153 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:52.954164 28138 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:52.977639 28157 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:53.051226 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:53.074252 25177 raft_consensus.cc:606] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:53.104703 25177 raft_consensus.cc:614] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:53.026810 28140 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:53.047966 27963 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.082106 23851 raft_consensus.cc:2368] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8.
I20260709 12:14:53.082322 23929 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:53.011787 28142 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:14:53.084913 28039 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:53.011262 28130 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.066211 28056 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.093169 23852 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:52.973726 28141 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 12:14:53.087473 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:53.073630 28081 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.025403 27880 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.105571 25171 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:53.019340 28118 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.109551 28143 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.164s
I20260709 12:14:53.109939 28144 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:53.047153 28121 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.110193 28162 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:53.105036 28160 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.108597 23859 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:53.112696 23781 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:53.113022 23783 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:53.114354 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64646.9us
I20260709 12:14:53.134353 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.135742 23777 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:53.136487 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.093005 28122 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.097234 27994 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:53.116367 25119 leader_election.cc:400] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8.
I20260709 12:14:53.138278 25119 leader_election.cc:403] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:14:53.100589 28084 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:53.142966 28152 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 12:14:53.122608 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 45215.4us
I20260709 12:14:53.101547 28153 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.119906 28161 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.121079 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:53.111562 28096 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.145025 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:53.141268 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.120116 28130 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:53.146145 23776 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:14:53.164537 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 43544.1us
I20260709 12:14:53.166383 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:53.167126 23784 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:53.128103 27949 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:53.171271 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.129048 28144 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.177381 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:53.173333 27949 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.179862 27949 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:53.184293 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68059.1us
W20260709 12:14:53.196647 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 58902.7us
I20260709 12:14:53.115010 28140 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.203652 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.112617 28157 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.141268 28084 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.074851 28145 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.206435 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:53.141247 27994 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.199772 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:53.209237 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.205977 23858 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:14:53.174399 28050 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:53.176332 27581 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:53.185067 28144 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.186251 28123 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.124128 27980 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.127533 28044 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:53.604866 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.346702 27923 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.172164 28161 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.378721 28145 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:53.140288 28162 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.263988 28163 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8.
W20260709 12:14:53.514628 28169 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:14:53.462486 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.529052 27880 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:53.379128 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 181298us
I20260709 12:14:53.639701 28153 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:53.641192 28153 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:53.643980 27880 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:53.644529 28093 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.645026 28093 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.647104 28093 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.348438 28039 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.473649 28075 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.347008 28112 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
W20260709 12:14:53.340862 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 414161us
W20260709 12:14:53.656150 28158 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.431s	user 0.001s	sys 0.000s
I20260709 12:14:53.408774 28171 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.639322 28053 consensus_meta.cc:287] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.859s	user 0.015s	sys 0.015s
I20260709 12:14:53.347575 28050 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.389297 28170 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:14:53.230283 28084 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:53.521171 28175 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.516297 28138 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:53.529225 28168 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:14:53.258245 28131 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:53.249334 28096 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:53.210080 28135 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:53.392086 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.707984 28053 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.352802 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.248214 27994 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:53.449813 28045 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.146220 28160 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.498445 28176 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:53.616065 25169 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 12:14:53.683015 28171 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.789788 25169 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:53.499878 27991 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.502130 28143 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:53.625761 27923 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.637228 28162 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:53.687130 23778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.032s	sys 0.016s
W20260709 12:14:53.796023 23778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.032s	sys 0.016s
W20260709 12:14:53.385156 28137 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.627s	user 0.016s	sys 0.002s
I20260709 12:14:53.700933 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.701303 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.674202 27581 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.660497 28157 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:53.732650 23855 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.003s
I20260709 12:14:53.733333 23851 raft_consensus.cc:2410] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:14:53.798659 23855 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
W20260709 12:14:53.735286 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.008s
W20260709 12:14:53.749975 23783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.043s	sys 0.144s
W20260709 12:14:53.799731 23783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.043s	sys 0.144s
I20260709 12:14:53.801175 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:53.704025 27974 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.644501 27880 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.808867 27581 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.809648 27923 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.811808 27880 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:53.435212 28141 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:53.770219 25176 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:53.812882 28141 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:53.771660 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.046s	sys 0.127s
W20260709 12:14:53.814136 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.046s	sys 0.127s
W20260709 12:14:53.814461 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 666274us
I20260709 12:14:53.781890 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:53.744264 28142 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.000s	sys 0.011s
I20260709 12:14:53.499023 28139 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.790629 25172 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:53.792376 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.744769 28140 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:53.700177 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:53.799573 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.004s	sys 0.008s
I20260709 12:14:53.801936 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:53.731658 23826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.007s	sys 0.002s
W20260709 12:14:53.825737 23826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.002s
I20260709 12:14:53.804713 28137 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.808429 27974 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.818817 25169 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:53.841650 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:53.821105 23784 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.002s
W20260709 12:14:53.817795 28109 consensus_meta.cc:287] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 1.076s	user 0.009s	sys 0.011s
I20260709 12:14:53.845806 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.793708 28176 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:53.828027 25113 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.030s
W20260709 12:14:53.850272 25113 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.014s	sys 0.030s
W20260709 12:14:53.850992 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 706003us
I20260709 12:14:53.824676 23856 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:53.792219 27989 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.797292 28143 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:53.822674 23853 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:53.843642 23784 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
W20260709 12:14:53.820523 28142 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.000s	sys 0.012s
I20260709 12:14:53.859851 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.862001 23855 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 6.
I20260709 12:14:53.821909 27991 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:14:53.853183 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:53.828809 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64633.4us
I20260709 12:14:53.849431 23785 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:53.865983 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.866719 23929 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:53.822511 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.861095 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.868369 23853 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:53.747421 28175 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.871291 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:53.868822 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:53.872550 23852 raft_consensus.cc:2410] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:14:53.829753 28183 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:53.873543 23856 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.875041 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:53.877995 23853 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:53.879423 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.786448 28053 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.881039 23855 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:53.845352 28140 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:53.878175 25177 raft_consensus.cc:606] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:53.860381 28143 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:53.849884 28157 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:53.864786 28050 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.857992 28176 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:53.892923 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:53.909436 23750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.008s
W20260709 12:14:53.909911 23750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.000s	sys 0.008s
I20260709 12:14:53.910445 23852 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:53.911940 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.917132 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:53.917910 23929 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:53.880767 28040 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:53.863257 27989 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:53.917534 28182 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:14:53.934895 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:53.940402 28137 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.941192 28180 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.941957 28180 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:53.944718 28193 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:53.945642 28193 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.950045 25176 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:53.932772 28138 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:53.973763 25176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:53.963579 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:53.979075 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:53.981081 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:53.974501 25176 raft_consensus.cc:606] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:53.983038 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:53.905696 28140 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.988360 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:53.961380 28141 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:53.912949 28157 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:53.997363 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:53.970419 28138 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:53.996115 23824 connection.cc:256] RPC call timeout handler was delayed by -1.0249859959999412s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 12:14:54.006229 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.965632 28137 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:53.998731 23751 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:54.010087 23929 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:54.009629 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:54.010855 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.014074 23852 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:14:54.015244 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.017331 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.018092 23852 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:53.987538 28040 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.021790 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.022586 25172 raft_consensus.cc:2393] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:54.023419 23853 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 7.
I20260709 12:14:54.026998 23780 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:54.032724 28172 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.034874 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:54.036405 28044 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:54.036947 28044 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.037506 28044 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:54.042935 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.044015 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.044689 23824 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.045782 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.046501 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:54.046509 23929 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:53.890566 28183 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.008332 28138 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.049690 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.052609 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.020915 28109 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:54.004040 27991 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:54.053462 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:54.047570 28155 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:14:54.063321 27974 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:53.934883 28080 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:14:54.063336 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.067577 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.068315 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.068413 28183 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.072124 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.072923 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.073189 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.045437 28135 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:54.051929 28039 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:54.060225 28162 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:54.021288 28137 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.060814 28109 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.060071 28170 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:54.053061 28186 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.024504 28141 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:14:54.047109 28050 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:14:54.095741 28192 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 12:14:54.079380 28145 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.083011 28153 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:54.075992 27581 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.090276 28162 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:54.102289 28056 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.967s	user 0.011s	sys 0.000s
I20260709 12:14:54.087632 28039 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.106484 23856 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:54.118074 23929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:14:54.102252 28109 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:54.128371 23929 raft_consensus.cc:606] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:54.126969 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 114651us
I20260709 12:14:54.128859 23929 raft_consensus.cc:614] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:54.111681 27880 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:14:54.133172 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:54.136559 28190 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:14:54.137329 28174 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.128204 28039 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.116778 28162 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.133031 28183 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:14:54.113322 28053 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.112767 28170 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:54.119551 28188 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.090418 28178 consensus_queue.cc:1059] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:14:54.139931 28181 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:54.171967 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.114295 28160 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.152199 27908 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.174899 23856 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:54.117439 28171 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.107909 28186 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.152439 28175 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:54.160882 28081 consensus_meta.cc:287] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.029s	user 0.015s	sys 0.001s
W20260709 12:14:54.172828 27949 consensus_meta.cc:287] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.992s	user 0.008s	sys 0.008s
W20260709 12:14:54.174517 28110 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.172s	user 0.016s	sys 0.000s
I20260709 12:14:54.188244 28203 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.178577 28188 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.185622 28181 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:54.187155 23929 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:14:54.130714 28056 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.195247 23929 pending_rounds.cc:85] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 4
I20260709 12:14:54.195513 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.195540 23929 pending_rounds.cc:107] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:14:54.151872 28174 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.179849 28193 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.197803 23929 raft_consensus.cc:2889] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.184446 27908 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.172087 28139 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:14:54.182039 28180 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.209069 28174 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:54.209354 28130 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.051s	user 0.008s	sys 0.005s
I20260709 12:14:54.209810 28130 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.210837 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.201815 28181 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.211884 28130 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.212538 28084 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.213740 23927 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:54.214358 23925 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:54.218372 28186 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:54.199092 27963 consensus_meta.cc:287] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.075s	user 0.013s	sys 0.002s
I20260709 12:14:54.222754 23926 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:54.226979 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.227566 23856 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 8: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:54.229555 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2678) took 1095 ms. Trace:
I20260709 12:14:54.194746 27949 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.195472 28081 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.229833 23779 rpcz_store.cc:276] 0709 12:14:53.134505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.134626 (+   121us) service_pool.cc:224] Handling call
0709 12:14:53.172704 (+ 38078us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:53.689372 (+516668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.689406 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:53.699508 (+ 10102us) log.cc:844] Serialized 26 byte log entry
0709 12:14:53.814087 (+114579us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:53.815142 (+  1055us) log.cc:844] Serialized 14 byte log entry
0709 12:14:53.892022 (+ 76880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:53.892151 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:54.229364 (+337213us) raft_consensus.cc:1685] finished
0709 12:14:54.229380 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.229504 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49543,"prepare.queue_time_us":5151,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":335453,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":23890,"thread_start_us":659153,"threads_started":3,"wal-append.queue_time_us":289241}
I20260709 12:14:54.228114 28157 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.212412 28209 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:14:54.179955 27991 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.239328 25168 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:54.215952 28203 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.241206 23928 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:54.231801 28137 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.225821 28170 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.239900 28142 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.224952 27963 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.253010 28188 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.256585 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.230881 28138 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:54.248761 28144 consensus_meta.cc:287] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.005s
I20260709 12:14:54.259259 28142 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.269791 27963 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.273450 28145 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.274492 27949 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.284837 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:54.276063 28208 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:14:54.291460 28127 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.294386 28138 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.258900 27994 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:54.248852 28176 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.303936 23855 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:54.306879 28176 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.308403 23932 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:14:54.309105 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 585) took 1232 ms. Trace:
I20260709 12:14:54.309407 23932 rpcz_store.cc:276] 0709 12:14:53.076761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.076887 (+   126us) service_pool.cc:224] Handling call
0709 12:14:54.307955 (+1231068us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b5800062a98. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68da22 00007fb2b35a68d9 00007fb2b2700ee0 00007fb2b27ca0ab 00007fb2b27ca131 00007fb2b278bfc0 00007fb2b2776929 00007fb2b27763d2 00007fb2abcedbd4
0709 12:14:54.308171 (+   216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:54.308825 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.309044 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8066944}
I20260709 12:14:54.288399 28183 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.312417 28081 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.312068 28183 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.313208 28174 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.313934 28174 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.316283 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.316598 27908 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.295447 27989 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.296830 28084 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.284634 28044 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.322696 27994 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:54.283846 27877 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.323725 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.323772 23852 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.326599 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1333) took 1149 ms. Trace:
I20260709 12:14:54.325345 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 604) took 1346 ms. Trace:
I20260709 12:14:54.329727 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.335309 28202 raft_consensus.cc:993] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:54.327976 28183 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.302126 28109 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.327105 27963 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.341969 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 3157) took 1489 ms. Trace:
I20260709 12:14:54.326973 27949 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.342775 27908 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.343792 23779 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:54.332222 23851 rpcz_store.cc:276] 0709 12:14:53.177012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.177139 (+   127us) service_pool.cc:224] Handling call
0709 12:14:54.326359 (+1149220us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54004137d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.326495 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253973888}
I20260709 12:14:54.333007 23857 rpcz_store.cc:276] 0709 12:14:52.978928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.979103 (+   175us) service_pool.cc:224] Handling call
0709 12:14:52.979449 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:53.610602 (+631153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.610640 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:53.610688 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:53.610722 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.325070 (+714348us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000a0fd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:54.325116 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.325235 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665577344}
I20260709 12:14:54.349246 23852 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:14:54.344204 23859 rpcz_store.cc:276] 0709 12:14:52.852713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.852831 (+   118us) service_pool.cc:224] Handling call
0709 12:14:54.341787 (+1488956us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400436cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.341901 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388113664}
I20260709 12:14:54.354058 27963 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.295083 28056 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.336018 28202 raft_consensus.cc:1081] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:14:54.281949 28144 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.366815 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.383451 23925 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:54.379328 28096 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.005s
I20260709 12:14:54.323558 28044 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.372968 23932 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:54.375739 27908 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.304888 28172 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.357070 27963 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.371829 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.384663 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:54.389786 27908 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.390545 28110 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:54.389235 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.391976 23929 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:54.393028 23856 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:54.342993 27949 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.395313 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:54.389158 28174 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.400694 28203 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.402887 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.402166 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.392023 27991 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:54.401481 23933 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:14:54.403884 23932 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:54.403919 23929 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:54.397621 27989 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.400571 28161 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:54.408320 25168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 378) took 1546 ms (client timeout 1993 ms). Trace:
I20260709 12:14:54.416203 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 382) took 1514 ms. Trace:
I20260709 12:14:54.440255 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2720) took 1590 ms. Trace:
W20260709 12:14:54.469231 25168 rpcz_store.cc:269] 0709 12:14:52.861469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.873783 (+ 12314us) service_pool.cc:224] Handling call
0709 12:14:54.408107 (+1534324us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001e87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.408218 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579793664}
I20260709 12:14:54.469841 25171 rpcz_store.cc:276] 0709 12:14:52.901965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.908217 (+  6252us) service_pool.cc:224] Handling call
0709 12:14:52.915643 (+  7426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:53.125581 (+209938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.415930 (+1290349us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54007611d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:54.416115 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431044480}
I20260709 12:14:54.473594 23778 rpcz_store.cc:276] 0709 12:14:52.849537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.026692 (+177155us) service_pool.cc:224] Handling call
0709 12:14:53.026998 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:53.046473 (+ 19475us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:53.116345 (+ 69872us) log.cc:844] Serialized 14 byte log entry
0709 12:14:53.941483 (+825138us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b180012b5c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5557 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:54.423622 (+482139us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b180012b5c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a
0709 12:14:54.423958 (+   336us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:54.424630 (+   672us) log.cc:844] Serialized 52 byte log entry
0709 12:14:54.424805 (+   175us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:54.425039 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.425253 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.439889 (+ 14636us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:54.439950 (+    61us) raft_consensus.cc:1685] finished
0709 12:14:54.439968 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.440155 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18989,"prepare.queue_time_us":158868,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":20711,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":16962,"spinlock_wait_cycles":11765504,"thread_start_us":683630,"threads_started":2,"wal-append.queue_time_us":661249}
W20260709 12:14:54.411265 28204 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.185s	user 0.003s	sys 0.000s
I20260709 12:14:54.405300 27991 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.433660 28118 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.452101 28166 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:54.409580 27989 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.452946 27999 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:54.413182 27908 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.421708 28181 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.414986 28144 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.409152 27949 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.410553 28163 raft_consensus.cc:3055] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:54.416980 28161 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.430724 28039 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:54.426857 28162 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.492811 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.480748 27991 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.483835 27989 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.489475 28166 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:54.490268 28161 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.513814 28166 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.487931 28163 raft_consensus.cc:740] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:54.521513 28181 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.523934 28039 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:14:54.524457 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.525846 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.526486 25174 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:54.526904 28166 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.527397 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:54.528005 25173 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.530537 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.531220 25174 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:54.526552 28039 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:54.532847 23781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2719) took 1683 ms (client timeout 1871 ms). Trace:
W20260709 12:14:54.533296 23781 rpcz_store.cc:269] 0709 12:14:52.849282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.932309 (+ 83027us) service_pool.cc:224] Handling call
0709 12:14:54.532685 (+1600376us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400109854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.532788 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330786176}
I20260709 12:14:54.536042 27949 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.536130 28181 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:54.536908 27949 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.539458 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.494215 28109 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.552561 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.553047 23931 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:54.534274 28143 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.009s
I20260709 12:14:54.494856 28080 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:54.490273 27999 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.560595 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.561607 25173 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.561762 25174 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:14:54.562157 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.564162 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.564924 23929 raft_consensus.cc:2368] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11.
I20260709 12:14:54.566093 23929 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:54.567095 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.561309 28080 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.533105 28039 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.572345 28118 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.586110 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.570946 28143 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.583758 28080 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.581933 28162 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.587119 27999 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.597047 28039 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.585376 28118 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.587903 27949 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:54.615335 28143 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.598372 27991 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.588042 28161 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:54.591826 28216 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:14:54.623675 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.608906 27853 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.579533 28163 consensus_queue.cc:260] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.610304 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.612233 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2708) took 2124 ms. Trace:
I20260709 12:14:54.621132 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.624459 25170 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:54.627936 23782 rpcz_store.cc:276] 0709 12:14:52.487675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.852512 (+364837us) service_pool.cc:224] Handling call
0709 12:14:52.878341 (+ 25829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:53.162467 (+284126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.162498 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:53.162542 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:53.162577 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.611966 (+1449389us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b540012bbd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:54.612010 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.612167 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652937984}
I20260709 12:14:54.630924 25174 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:14:54.633638 25172 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:54.640005 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.640681 23929 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:14:54.639213 28124 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.641129 27877 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.632738 28049 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.643955 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.644984 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.645586 23781 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.630053 27853 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.646150 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.646317 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.646771 23778 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:54.646910 23782 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.645761 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.648284 23779 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:54.649241 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.649401 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:14:54.649937 23781 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:54.650354 23781 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.651003 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.650038 23779 raft_consensus.cc:2410] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:54.651659 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.652428 23776 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:54.642390 28049 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.651377 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.653995 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.653357 28173 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.641746 27877 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.655969 28173 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.656220 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.657264 23780 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.631498 28109 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.658542 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.617565 28121 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.660126 23776 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.654685 23783 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:14:54.660941 23858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 605) took 1681 ms (client timeout 1912 ms). Trace:
I20260709 12:14:54.667104 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.647447 28163 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:54.658499 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.670288 23779 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:54.671809 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.672729 23779 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:54.669636 23858 rpcz_store.cc:269] 0709 12:14:52.979356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.979630 (+   274us) service_pool.cc:224] Handling call
0709 12:14:54.660768 (+1681138us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400386654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.660881 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479462144}
I20260709 12:14:54.671657 23929 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:54.674379 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.674783 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.675360 23929 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:14:54.676126 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.680444 23782 raft_consensus.cc:1240] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:54.653369 28143 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:54.681028 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.681263 23783 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 7: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:54.681293 23776 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:54.667536 23781 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.686005 23778 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:54.683744 23780 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:54.641675 28124 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.686048 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.689978 23783 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:54.692040 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.692579 23783 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.687557 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.690944 23781 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.693475 23780 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.688613 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.694754 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.695366 23776 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:54.653539 27989 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.697160 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.697597 23779 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.689802 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.681602 28134 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:54.712618 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.713413 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.698336 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.714205 23782 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.669894 28109 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.731027 23929 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:54.712294 28219 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
I20260709 12:14:54.696264 27877 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.707104 28198 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:14:54.723914 28124 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.724643 28049 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.725500 28142 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.728757 28173 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.729722 27853 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.744594 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.745438 28142 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:54.745963 28142 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.746412 28142 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.733755 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.732774 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:54.748106 23929 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:14:54.749431 23929 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:54.750733 23929 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:54.713358 28134 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:54.739409 28118 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:54.739845 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.753571 27949 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:54.741178 28109 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.732107 28143 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.743330 23930 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:54.741118 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.754601 23926 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.756093 23933 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:14:54.756894 25119 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 1.912s (SENT) [suppressed 5 similar messages]
I20260709 12:14:54.740639 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.758365 25177 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:54.697809 23776 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.761471 23780 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:54.768954 23782 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:54.769497 23926 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:54.769668 23776 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.773180 23783 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:54.779307 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2705) took 1929 ms. Trace:
I20260709 12:14:54.773970 28092 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:54.779844 23934 rpcz_store.cc:276] 0709 12:14:52.849872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.850020 (+   148us) service_pool.cc:224] Handling call
0709 12:14:52.850337 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:53.100540 (+250203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.100570 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:53.100610 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:53.100638 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.779043 (+1678405us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54002b6454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:54.779098 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.779238 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574953856}
W20260709 12:14:54.810799 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 78120.5us
I20260709 12:14:54.815105 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1244) took 1112 ms. Trace:
I20260709 12:14:54.810987 28124 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:54.822808 28222 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.134s	user 0.000s	sys 0.003s
I20260709 12:14:54.786125 28092 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:54.824311 27989 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.787484 23898 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.828253 23927 rpcz_store.cc:276] 0709 12:14:53.702797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.892642 (+189845us) service_pool.cc:224] Handling call
0709 12:14:54.814868 (+922226us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002436d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.815026 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698435328}
I20260709 12:14:54.842628 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 784) took 2127 ms. Trace:
W20260709 12:14:54.861407 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 74024.8us
I20260709 12:14:54.860479 23785 rpcz_store.cc:276] 0709 12:14:52.715723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:52.750415 (+ 34692us) service_pool.cc:224] Handling call
0709 12:14:54.842423 (+2092008us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400093b54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.842564 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521753344}
I20260709 12:14:54.844955 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:54.850839 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.851677 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.861764 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.862969 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:54.863541 23853 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.865124 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:54.849349 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.863541 23859 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:54.867698 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.866760 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.868343 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.868969 23859 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:54.869287 23898 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.870637 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.859721 28124 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:54.900460 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:54.872910 28118 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:54.874414 28032 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:54.874797 28045 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.859309 28157 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.782672 28134 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.939211 28157 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.939661 28134 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.940897 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:54.941573 28157 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.942090 23900 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:54.943039 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:54.857352 28211 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:14:54.881321 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.884637 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.944289 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.886027 28173 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:54.886837 28198 raft_consensus.cc:2764] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:54.889845 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.893430 27991 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.876683 28092 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.896689 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.962033 23857 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:54.897423 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.826280 28080 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:54.896142 28050 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.900665 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:54.901091 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:54.970986 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:54.903273 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.866856 28122 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.928869 28118 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.913130 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:54.977995 25169 raft_consensus.cc:606] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:54.978433 25169 raft_consensus.cc:614] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:54.913947 28161 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.916988 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1334) took 1739 ms. Trace:
I20260709 12:14:54.903944 28049 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 12:14:54.929889 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 59326.5us
I20260709 12:14:54.933028 23853 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:54.934448 28140 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:54.935986 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.962715 27974 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.937381 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:54.985677 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 85089.7us
I20260709 12:14:54.986008 23927 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.938019 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.986605 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:54.944633 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44262.1us
I20260709 12:14:54.987044 23931 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.946725 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:54.952522 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:54.952862 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.952754 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:54.960214 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 203336us
I20260709 12:14:54.989677 23855 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:54.988317 23859 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:14:54.957386 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.987485 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:54.966837 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:54.969797 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.883483 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.992722 23860 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.967526 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.993986 23933 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:54.875962 27853 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:54.970970 28050 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.008404 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.008064 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.009783 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:55.010388 23859 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:54.977312 23824 leader_election.cc:400] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11.
I20260709 12:14:54.938973 28032 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:54.971333 28131 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:54.979507 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:54.979516 23854 rpcz_store.cc:276] 0709 12:14:53.177327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.177446 (+   119us) service_pool.cc:224] Handling call
0709 12:14:54.660895 (+1483449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:54.661073 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:54.661100 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:54.661142 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:54.661170 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.661222 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:54.916795 (+255573us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400386650. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:54.916921 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3340482944}
I20260709 12:14:54.936714 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:54.984859 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.045010 23932 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:54.966395 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:55.045675 23926 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:54.959860 28183 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:54.972863 28093 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:54.938910 28112 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:14:54.999912 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:54.968513 27949 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:55.075714 23858 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:55.039016 28050 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.979668 28039 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:14:54.980119 28092 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.979672 28118 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:55.043284 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.045235 23853 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:55.053023 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.064808 28093 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:55.054816 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 67408us
I20260709 12:14:54.985720 27974 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.995498 28172 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:54.994457 28143 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:55.060359 28183 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:55.071198 23860 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:55.012310 23824 leader_election.cc:403] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:14:55.107281 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 132480us
I20260709 12:14:55.109387 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:55.071254 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 84741.7us
I20260709 12:14:55.072785 27991 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.072649 27950 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.938921 28045 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.085363 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.042691 27853 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.101979 27974 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.044128 28131 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:55.122339 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 164727us
I20260709 12:14:55.090592 23855 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 7.
I20260709 12:14:55.051934 28173 consensus_queue.cc:1243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:55.098444 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:55.110173 28183 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:55.102597 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.078128 28040 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:55.008553 28109 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.077945 28164 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:55.077226 28124 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.112886 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:14:55.115118 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.112244 28157 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11.
I20260709 12:14:55.116050 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.124986 28131 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:55.121035 23856 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:55.108392 28142 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:54.984660 28161 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.095355 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:55.088470 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.129281 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.129526 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.119442 28045 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.111482 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.138759 25169 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:55.141137 23853 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:55.097783 28093 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:55.142906 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:55.175921 25169 raft_consensus.cc:614] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:55.144582 28177 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:55.119240 27950 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:55.149888 28224 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:14:55.119570 28170 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:14:55.133239 28143 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:55.156010 23854 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:55.131016 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:55.182885 23852 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 7: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:55.138339 28040 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:55.169183 23927 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:55.170713 23931 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:55.120009 28044 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 12:14:55.173720 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 78458.6us
I20260709 12:14:55.173143 28039 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:55.167281 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:55.181133 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 69724.3us
I20260709 12:14:55.203807 23852 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.204077 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.120452 28032 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.135800 28164 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:55.199542 28227 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:14:55.182448 28143 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.205502 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.177145 27925 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:55.201704 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:55.179402 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:55.208081 23859 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:14:55.221055 23859 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:55.178504 28112 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:55.184733 28173 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.212738 23853 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:14:55.204348 23856 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:55.177590 28177 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:55.215559 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48390.7us
I20260709 12:14:55.224009 23858 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:14:55.227063 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.229076 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:55.223384 23859 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.270200 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:55.328866 23900 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.325790 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:55.329747 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59683.3us
I20260709 12:14:55.316411 23860 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:55.206012 28080 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:55.209164 28172 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.214984 28164 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.183261 28162 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 12:14:55.352129 28126 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:55.352809 28126 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.355172 28126 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.356626 28056 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:55.357188 28056 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:55.358038 28056 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.316798 28198 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.360872 23853 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:55.361937 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2722) took 2225 ms. Trace:
I20260709 12:14:55.340552 27925 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:55.369151 27925 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.219213 28143 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:55.316825 28163 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.334034 28232 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:55.334265 28140 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:55.362317 23777 rpcz_store.cc:276] 0709 12:14:53.136624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.157549 (+ 20925us) service_pool.cc:224] Handling call
0709 12:14:53.157948 (+   399us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:53.174479 (+ 16531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:53.174515 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:53.176599 (+  2084us) log.cc:844] Serialized 52 byte log entry
0709 12:14:54.317965 (+1141366us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:14:54.741900 (+423935us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b1800426b80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:14:54.844379 (+102479us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:54.924794 (+ 80415us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.945377 (+ 20583us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.981183 (+ 35806us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b1800426b80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:14:54.981424 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.981520 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:55.361706 (+380186us) raft_consensus.cc:1685] finished
0709 12:14:55.361728 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:55.361849 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112048,"prepare.queue_time_us":80491,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":170191,"raft.run_cpu_time_us":9692,"raft.run_wall_time_us":1055632,"spinlock_wait_cycles":216130688,"thread_start_us":102278,"threads_started":1,"wal-append.queue_time_us":1768466}
I20260709 12:14:55.391891 28040 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.341051 28121 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:55.331864 28096 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:55.398990 28225 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:14:55.362808 27974 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.378813 28112 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:55.474287 27974 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.399720 28232 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.403223 28140 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.411974 28121 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:55.476774 28230 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:14:55.517243 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 288352us
I20260709 12:14:55.501448 28172 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:55.518834 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 339457us
W20260709 12:14:55.520994 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 192172us
I20260709 12:14:55.522437 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.453816 28096 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.522284 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.508356 28112 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:55.515373 28121 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.531332 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.516135 28140 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:55.537089 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:55.537123 23928 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:55.538602 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:55.539276 23927 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:55.513350 28232 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:55.548133 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.550006 23927 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:14:55.552865 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:55.553504 25177 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.540653 27974 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:55.541076 28142 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:55.557421 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:55.530906 28039 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:55.561364 23928 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.524121 28172 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.562492 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:55.509101 28177 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.565006 28112 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:55.579821 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 236144us
I20260709 12:14:55.576997 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:55.572871 28134 consensus_meta.cc:287] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.633s	user 0.007s	sys 0.005s
I20260709 12:14:55.563838 28044 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.565186 28123 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:55.609122 25176 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:14:55.596217 28228 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:14:55.558179 28093 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.569672 28211 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:14:55.605386 28123 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:55.628582 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 72656.6us
I20260709 12:14:55.610028 27853 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.651916 23931 raft_consensus.cc:2368] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:55.600495 27877 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:14:55.626291 28044 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.630249 28123 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:55.628278 28244 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:55.663344 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 80378.3us
I20260709 12:14:55.699870 28096 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:55.709475 28140 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.712333 28140 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.694883 28080 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.717447 28080 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.721277 28080 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:55.724068 28237 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:14:55.725744 28245 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:14:55.726672 28250 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:14:55.732225 23777 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:14:55.732625 23777 pending_rounds.cc:85] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 9
I20260709 12:14:55.732839 23777 pending_rounds.cc:107] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:14:55.733043 23777 raft_consensus.cc:2889] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:55.733772 28092 consensus_meta.cc:287] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.633s	user 0.008s	sys 0.001s
I20260709 12:14:55.668998 28044 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.669726 28211 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:55.736377 28211 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:55.737365 28211 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.737697 27991 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:14:55.739302 28251 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:14:55.744074 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.747875 25171 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:55.751231 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:55.664883 28247 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:55.750576 27974 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:55.753050 28045 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.003s
I20260709 12:14:55.769472 28040 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:55.797132 27974 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:55.758622 28118 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.655s	user 0.004s	sys 0.007s
I20260709 12:14:55.801615 28141 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:55.804101 28177 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:55.794989 28247 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:55.744865 28044 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:55.769825 28183 consensus_meta.cc:287] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.002s
I20260709 12:14:55.749297 28123 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:55.797744 28248 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:14:55.830698 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:55.833230 28131 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.002s
I20260709 12:14:55.662117 28039 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.820513 27877 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:55.818609 25177 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:55.811410 28164 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.834537 28172 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.821594 28141 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:55.850157 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.821429 28118 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:55.861490 28050 consensus_meta.cc:287] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.002s
I20260709 12:14:55.862394 28164 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.862638 23927 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:14:55.862393 28118 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.863838 28123 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.861956 28050 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.864228 28143 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.866487 28143 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:55.867266 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:55.866097 28050 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.848264 28093 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:55.848847 27877 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.876035 27877 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:55.834822 27989 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:14:55.845086 27991 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:55.830974 28045 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.879431 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:55.893679 25177 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:55.833356 28183 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.897995 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.887485 28141 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:55.902769 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 73490.8us
W20260709 12:14:55.902931 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 151788us
I20260709 12:14:55.904328 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.906093 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:55.906435 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:55.833087 28247 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:55.909622 23928 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:55.910866 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:55.913944 23933 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:55.849232 28131 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.918414 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:55.919672 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:55.916944 23859 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:55.888752 28261 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:55.892843 27974 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:55.931427 23857 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:55.967229 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:55.982817 25177 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:14:55.984862 28257 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:55.990990 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1363) took 1084 ms. Trace:
I20260709 12:14:56.000434 23858 rpcz_store.cc:276] 0709 12:14:54.906348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.166372 (+260024us) service_pool.cc:224] Handling call
0709 12:14:55.223777 (+ 57405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.328330 (+104553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:55.990761 (+662431us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54002dfd50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:55.990902 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72974208}
I20260709 12:14:55.994462 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.009531 28183 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:55.930233 28093 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:56.010802 28093 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:56.012215 28265 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:14:56.013185 28093 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.930431 28255 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 12:14:55.903985 28258 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:56.015676 28258 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.016057 28258 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.996351 27974 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:56.020257 25169 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:56.020623 27974 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.022293 28163 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:56.022810 28163 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:55.928396 28109 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:55.931360 28266 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.024926 28163 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.024814 28266 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.028627 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.029763 28247 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.030946 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:56.031314 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 125989us
I20260709 12:14:56.031621 25175 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.031989 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.032503 28045 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:55.996604 28241 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:56.034068 28109 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:56.034345 28241 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.036530 28241 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.037024 28241 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.037495 28241 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.039270 25175 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:14:56.039938 28121 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.041388 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.041708 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.042128 28121 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.042383 25168 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.907410 28238 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 12:14:56.032277 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.050457 28131 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:56.050760 28164 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:55.901448 28044 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:14:56.052850 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.054327 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.058251 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.085608 25168 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:14:56.089105 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67616.4us
I20260709 12:14:56.097828 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.051234 28164 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:14:56.110545 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 56304.1us
I20260709 12:14:56.111296 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.111774 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.112326 23854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:14:56.113006 23854 raft_consensus.cc:606] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.113385 23854 raft_consensus.cc:614] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:56.116130 28232 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.001s
I20260709 12:14:56.116593 28232 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.114313 23898 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:55.967307 28123 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.143s
I20260709 12:14:56.146420 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:56.172336 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 58135.5us
I20260709 12:14:56.171399 28049 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:56.173919 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:56.174175 28049 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.174255 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.175438 25175 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:56.179576 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:56.180302 23853 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:56.174691 28049 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:56.073138 28260 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:56.183773 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.098282 28109 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:56.188113 28238 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:56.179116 28198 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:14:56.190315 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:14:55.975286 28261 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.198298 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.198907 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.200613 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.205469 23779 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:56.200748 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.201905 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.096352 28134 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.109695 28240 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.118809 28232 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.207877 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.208117 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.214306 23779 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.214812 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.215624 23776 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.206091 23781 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:14:56.216112 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.206492 23778 raft_consensus.cc:2368] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:56.029711 27877 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.188014 28172 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:56.209048 23784 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.195415 28239 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.109954 28121 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:56.211347 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.205480 28269 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:56.110399 28163 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.218261 23781 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:56.181020 28264 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:56.175185 28164 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:56.218796 23778 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:56.219208 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.219663 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.232107 23779 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.195236 28050 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 12:14:56.232972 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.232343 23780 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.235633 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.229980 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.217473 28177 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:56.213340 23782 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:56.211165 28241 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.225010 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.205503 27853 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:14:56.238956 23781 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.227228 28163 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.239862 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.203632 28260 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:56.223889 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64928.8us
W20260709 12:14:56.244434 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.236238 23778 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.248112 28266 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:56.248374 28271 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:14:56.249464 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.253120 28260 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.214422 28134 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.225677 28239 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.268962 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.269627 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.270272 23783 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.271313 23780 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:56.271617 23780 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:56.272758 25169 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:56.276726 28211 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.279949 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.280910 23783 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:14:56.281255 23784 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:56.281716 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.282186 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.279970 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.283262 23785 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.284041 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.284548 23785 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:14:56.285254 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.286176 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.287106 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.245654 28177 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.288822 27877 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.289991 28258 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.290159 23780 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.291200 23780 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:56.292309 28177 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.292565 28261 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.294283 28163 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.246026 28264 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:56.295518 28264 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.282403 23783 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.298943 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.297839 28134 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.319192 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.321362 23928 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:56.300105 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.300689 23781 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:14:56.322455 23784 raft_consensus.cc:2368] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:56.323289 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.215564 28240 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.306129 28239 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:56.311913 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.325582 23925 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.287220 28049 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.314458 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.315606 23785 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:56.316738 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.349942 23933 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:14:56.317729 23783 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:56.320617 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:56.325932 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 121151us
W20260709 12:14:56.313359 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45244.1us
I20260709 12:14:56.330551 23928 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:56.322477 23778 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:14:56.335186 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.359263 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.360054 23778 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.361966 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.362581 23778 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:14:56.364789 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.365556 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.366441 23778 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
W20260709 12:14:56.367539 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.396637 23779 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
W20260709 12:14:56.397025 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 57776.2us
I20260709 12:14:56.398077 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.398887 25175 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.402726 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.403447 23778 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
W20260709 12:14:56.404824 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51473.9us
I20260709 12:14:56.405405 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.406649 23783 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:14:56.407222 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.404943 23780 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:14:56.408025 23780 pending_rounds.cc:85] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 9
I20260709 12:14:56.408265 23780 pending_rounds.cc:107] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:14:56.408486 23780 raft_consensus.cc:2889] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:56.410573 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:56.413580 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.416035 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.416903 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.417207 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.417764 25168 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:56.309860 28258 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.422646 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.427058 23784 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:56.427423 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.427876 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.423997 23898 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:56.429044 23930 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.437849 23930 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:56.440678 23925 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:56.446744 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.448321 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.448913 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.449919 25177 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:14:56.449196 25172 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:56.418853 27955 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.452035 25175 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:56.454968 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:56.452484 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.426994 28198 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:56.420568 27999 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.426734 28050 raft_consensus.cc:2764] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:56.460493 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.444152 27853 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:56.466998 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.468039 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.426568 28239 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:14:56.448307 28257 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:56.456367 28039 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.462057 27974 raft_consensus.cc:2764] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:14:56.554996 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 94589.3us
I20260709 12:14:56.556807 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.558903 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.449412 28258 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:14:56.466653 27877 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:56.567809 25172 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:56.571874 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.576349 23930 raft_consensus.cc:2368] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
I20260709 12:14:56.555774 28050 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:56.459343 27955 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.505124 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:56.567358 25168 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:56.556627 28134 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:56.585527 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 80489.2us
I20260709 12:14:56.495281 28173 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:14:56.587212 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.467386 27999 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.566910 28261 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:56.590626 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.587846 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.595846 23783 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:56.596797 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.597420 23783 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:56.568924 28280 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.584036 28122 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:56.581955 28166 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:56.591097 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.587774 23930 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:14:56.591375 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.606823 25177 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:14:56.580878 28281 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.563570 28039 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:56.607259 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 182343us
I20260709 12:14:56.558588 28257 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.605525 23778 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.593552 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 2742) took 1056 ms. Trace:
I20260709 12:14:56.608254 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:56.612807 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.613299 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.588593 28157 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:56.590659 27877 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:56.603314 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:14:56.501997 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55353.7us
I20260709 12:14:56.613538 23780 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.604779 25172 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:14:56.613797 23930 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:56.624560 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.624749 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.625731 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.626838 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.627337 23780 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.612818 23931 rpcz_store.cc:276] 0709 12:14:55.536733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.536870 (+   137us) service_pool.cc:224] Handling call
0709 12:14:56.593373 (+1056503us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54000f78d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:56.593495 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233645312}
I20260709 12:14:56.628798 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.630023 28163 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:56.630391 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:56.631244 28163 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:56.631704 28163 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.632114 28163 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.640720 23780 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:14:56.622879 28239 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:56.642743 28239 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:56.643296 28239 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:56.642722 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.644292 28239 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.645984 28239 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.649725 28050 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.650308 28050 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.623093 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.615087 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.652748 28050 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.653381 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.655162 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.628810 23783 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:56.656327 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:56.657594 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.658211 23783 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.666170 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.667651 28261 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:56.676008 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52989.6us
I20260709 12:14:56.678308 23778 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:14:56.682739 28163 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.684553 28261 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:56.688577 28258 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:56.695077 27853 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:56.621937 28157 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:56.697296 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.598701 28280 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.679452 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:56.603648 28122 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:56.603900 28282 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:56.717521 23783 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:14:56.724009 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 218309us
I20260709 12:14:56.611334 28166 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:56.725257 23778 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:56.727284 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3187) took 1831 ms. Trace:
W20260709 12:14:56.726691 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 298870us
I20260709 12:14:56.732030 23856 rpcz_store.cc:276] 0709 12:14:54.896653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.152547 (+255894us) service_pool.cc:224] Handling call
0709 12:14:55.152846 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:55.325285 (+172439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.726984 (+1401699us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b54002d37d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:56.727218 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105176960}
I20260709 12:14:56.733536 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:56.744655 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47448.3us
I20260709 12:14:56.745646 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.747915 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:56.717216 27877 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:56.750376 25177 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:56.751151 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.754910 23825 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.755576 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:56.756069 23931 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:56.752729 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.760321 23824 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:56.762606 23931 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:56.752050 28261 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.729923 28240 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.764922 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.743162 28049 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.723788 28157 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.781769 28157 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:56.778151 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.784425 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:56.767867 28239 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:14:56.747802 28282 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:56.612829 28281 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.784576 23856 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:14:56.756592 28258 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:56.779460 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.757788 27853 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:56.802444 27955 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:56.771457 28261 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.804199 27999 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.783334 28173 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.804951 28173 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.809372 28258 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:56.810391 28258 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:56.810834 28258 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:56.818536 23858 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:56.800552 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.777896 28122 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.836083 28298 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:56.836606 28122 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.848475 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.849349 25177 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:56.855564 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 110393us
W20260709 12:14:56.858263 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.832445 28295 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:56.843037 28286 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:14:56.842869 28285 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:14:56.826095 28292 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:56.826236 28289 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:56.891270 23779 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:56.843037 28233 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:56.900305 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145447us
I20260709 12:14:56.887854 27950 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.893142 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:14:56.832396 28290 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:56.856814 27974 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:56.904963 25177 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:56.905074 23779 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:14:56.907882 23779 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:14:56.889889 28295 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:56.916519 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.826102 28166 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.922164 25114 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:14:56.901968 28233 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:56.918002 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.912983 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:56.929730 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 263630us
I20260709 12:14:56.844151 28258 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:56.906219 28290 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:56.918741 28295 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:56.934401 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2747) took 1373 ms. Trace:
I20260709 12:14:56.939795 28257 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:56.941911 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.888187 27877 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:56.958283 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.925136 28233 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:56.903272 27853 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:56.947927 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.947978 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.956548 23926 rpcz_store.cc:276] 0709 12:14:55.560948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.561422 (+   474us) service_pool.cc:224] Handling call
0709 12:14:55.577990 (+ 16568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.578347 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:55.578377 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:56.299114 (+720737us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b2c000bb9c8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:56.299504 (+   390us) log.cc:844] Serialized 26 byte log entry
0709 12:14:56.565678 (+266174us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:56.565730 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.565810 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:56.925192 (+359382us) raft_consensus.cc:1685] finished
0709 12:14:56.925223 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:56.934327 (+  9104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46571,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9552,"spinlock_wait_cycles":4292480,"thread_start_us":259728,"threads_started":1,"wal-append.queue_time_us":417242}
I20260709 12:14:56.952669 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:56.961405 23857 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:14:56.957873 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 197607us
I20260709 12:14:56.964542 23824 leader_election.cc:400] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:56.964892 23824 leader_election.cc:403] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:56.929438 28292 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:56.927893 28039 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:56.962276 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.972910 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:56.927713 28173 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.986344 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:56.986696 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:56.977523 23857 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:14:56.971639 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 85165us
I20260709 12:14:56.989557 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:56.979566 23825 leader_election.cc:400] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:56.991523 23825 leader_election.cc:403] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:56.931484 28109 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:56.928622 28240 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:56.941190 28290 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.994503 23825 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:56.994908 23749 leader_election.cc:400] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
I20260709 12:14:56.997676 23749 leader_election.cc:403] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:14:56.972877 28198 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:56.940821 28049 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:14:56.940999 28050 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:14:57.014127 28292 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.019125 23857 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:14:56.959061 28183 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:14:57.028208 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:57.031704 28301 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:14:56.996690 28240 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:57.043043 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 63051.2us
I20260709 12:14:57.044970 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:57.006151 28109 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
I20260709 12:14:56.988780 27955 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:57.000013 28173 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:56.993134 28039 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.001212 28308 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:57.048393 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:57.047580 25176 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 612) took 2146 ms (client timeout 2307 ms). Trace:
I20260709 12:14:57.047385 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.034400 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:57.059540 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65107.5us
I20260709 12:14:57.066061 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.066609 23933 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:57.067863 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:56.958700 28284 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 12:14:57.074530 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.043488 28240 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.074682 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:57.073169 25176 rpcz_store.cc:269] 0709 12:14:54.894489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.897153 (+  2664us) service_pool.cc:224] Handling call
0709 12:14:57.038383 (+2141230us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400820354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:57.040781 (+  2398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486381440}
I20260709 12:14:57.081270 23857 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:57.082897 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.085080 23926 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:57.082238 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.098266 23857 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:14:57.099910 23857 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:57.101394 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:57.109649 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:57.109498 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 110731us
I20260709 12:14:57.070217 28314 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:57.079457 28307 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:57.045267 28304 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
I20260709 12:14:57.080535 28170 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13.
W20260709 12:14:57.100625 28296 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:14:57.069392 28198 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:14:57.083011 28281 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:56.906787 28282 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.044721 28302 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:14:57.082165 28299 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:57.128072 28288 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.208s	user 0.001s	sys 0.001s
I20260709 12:14:57.074095 28039 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.137636 23857 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:14:57.138353 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 69359.5us
I20260709 12:14:57.072844 28309 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:14:57.114728 28314 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:57.072080 28308 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.133380 28307 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:57.160486 27989 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.155412 28316 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:57.086983 27853 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:57.173067 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 59792.2us
I20260709 12:14:57.172887 27989 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.161731 28239 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:57.175134 28239 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:57.175630 28239 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:57.176537 28211 consensus_meta.cc:287] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.899s	user 0.009s	sys 0.003s
I20260709 12:14:57.178519 28308 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.151330 28299 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.181231 28299 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.151435 28311 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:57.184470 28311 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:57.185403 28311 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:14:57.187976 23854 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.135689 27950 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:57.137889 27999 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:57.193255 27999 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.195011 28305 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:57.195724 28305 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.163954 28314 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:57.198339 28305 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.183899 28307 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.189136 28039 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:57.148837 28161 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.205221 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1374) took 2245 ms. Trace:
W20260709 12:14:57.207028 28164 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.944s	user 0.011s	sys 0.002s
W20260709 12:14:57.208614 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 163712us
I20260709 12:14:57.212175 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:57.220115 23854 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:57.222434 23776 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:14:57.223428 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 65357.2us
I20260709 12:14:57.208968 23852 rpcz_store.cc:276] 0709 12:14:54.960148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.380510 (+420362us) service_pool.cc:224] Handling call
0709 12:14:55.389615 (+  9105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:55.389847 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:55.533997 (+144150us) log.cc:844] Serialized 14 byte log entry
0709 12:14:57.204786 (+1670789us) spinlock_profiling.cc:243] Waited 4.81 ms on lock 0x7b18001c8dc0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287
0709 12:14:57.204885 (+    99us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:57.204935 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:57.204980 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.205048 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:57.205165 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9218,"prepare.queue_time_us":270112,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":101699,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":11061120,"thread_start_us":292076,"threads_started":3,"wal-append.queue_time_us":264289,"wal-append.run_cpu_time_us":1060,"wal-append.run_wall_time_us":1002845}
W20260709 12:14:57.234833 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 61045.7us
I20260709 12:14:57.237198 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:57.238881 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 205253us
I20260709 12:14:57.238973 28335 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:57.239810 28335 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:57.240788 28335 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.243911 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.248355 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:14:57.249744 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 175267us
I20260709 12:14:57.251317 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:14:57.252225 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:57.254935 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.256384 28334 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.257118 28039 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:57.251806 28327 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:14:57.260154 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:57.222370 28164 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.261605 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.263448 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:57.270090 23779 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:57.275441 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:57.273617 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:57.278030 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.269795 28321 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.316978 27950 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:57.270048 28039 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:57.319152 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:57.279227 28334 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:57.320670 23780 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.322115 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.322887 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.317397 28314 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.326071 23855 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:57.279824 28302 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:57.280550 28294 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:57.328624 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:57.337291 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:57.337947 23779 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:57.338512 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:57.339006 23779 raft_consensus.cc:2410] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:57.329818 23933 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 9: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:57.328528 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.339753 23852 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:14:57.334398 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.323441 28313 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:57.349327 23855 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:57.341300 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:57.354888 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.354941 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:57.363226 23780 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:57.327181 28039 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:57.365512 23926 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:57.366334 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.367350 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.368559 23852 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:57.334606 27950 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:57.369011 28039 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:57.368505 28294 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:57.334344 28334 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.418776 27950 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.396068 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:57.375447 28166 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:14:57.440037 28239 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.433509 28333 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:57.488131 28342 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:14:57.490427 28339 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:14:57.510581 28313 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.478664 28333 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:57.518388 28274 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.427s	user 0.003s	sys 0.000s
W20260709 12:14:57.505287 28289 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.598s	user 0.012s	sys 0.000s
I20260709 12:14:57.504309 28239 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:57.500841 28164 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:57.470621 28290 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:57.530769 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.531766 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:57.527655 28295 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 12:14:57.520036 28333 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:57.539351 28295 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.541469 28334 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.545056 23856 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:57.552551 28341 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.160s	user 0.002s	sys 0.000s
I20260709 12:14:57.529758 28289 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:57.569331 28141 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.483760 28294 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.585747 28295 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:57.586818 23925 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:57.587145 23859 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
W20260709 12:14:57.587874 28233 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.007s
I20260709 12:14:57.588469 28233 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.589910 23934 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:57.592850 27999 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:57.596516 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.599140 23859 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:14:57.600183 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:57.596753 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:57.601338 23779 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:57.570845 28157 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.789s	user 0.013s	sys 0.000s
I20260709 12:14:57.617651 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2714) took 1341 ms. Trace:
I20260709 12:14:57.618952 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:57.621001 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.623735 23784 rpcz_store.cc:276] 0709 12:14:56.276530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.397331 (+120801us) service_pool.cc:224] Handling call
0709 12:14:56.397624 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.427454 (+ 29830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.617351 (+1189897us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b540035a7d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:57.617579 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67276032}
I20260709 12:14:57.630793 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:57.632920 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.633661 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.637550 28239 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:57.615970 27999 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:57.648375 28266 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:57.649089 28266 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:57.654974 28277 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 1.467s	user 0.005s	sys 0.002s
I20260709 12:14:57.656901 28039 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.664645 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.650524 28266 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.671818 23779 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:14:57.648849 28290 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:57.674108 28321 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.643177 27999 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:57.733530 23779 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:14:57.674386 28049 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:57.750363 28039 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:57.743544 28124 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.752209 28124 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.744096 28131 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.761129 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.761236 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:14:57.763790 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 363373us
I20260709 12:14:57.764019 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.741923 27999 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:57.766602 25177 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:14:57.772294 23854 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:57.774602 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:57.742447 28233 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:57.788007 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:14:57.788983 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.747364 28049 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:57.795301 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.755543 28124 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:57.802778 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:57.806327 23853 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:57.804622 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:14:57.809649 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145071us
I20260709 12:14:57.810459 23825 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.811646 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:14:57.812783 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.814218 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 400) took 1253 ms. Trace:
I20260709 12:14:57.814639 25175 rpcz_store.cc:276] 0709 12:14:56.561190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.561353 (+   163us) service_pool.cc:224] Handling call
0709 12:14:56.564248 (+  2895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.605417 (+ 41169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:56.605445 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:56.605484 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:56.605525 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.813962 (+1208437us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b5400703354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:57.814015 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:57.814148 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94236416}
I20260709 12:14:57.816932 25175 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:57.822530 28238 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:57.751497 28333 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.781946 28260 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:57.830760 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2745) took 2274 ms. Trace:
I20260709 12:14:57.836464 23927 rpcz_store.cc:276] 0709 12:14:55.556795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.557012 (+   217us) service_pool.cc:224] Handling call
0709 12:14:55.557345 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:55.863056 (+305711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.830426 (+1967370us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54000f87d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:57.830636 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701549824}
I20260709 12:14:57.837462 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:14:57.830567 25169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 615) took 1800 ms (client timeout 1780 ms). Trace:
W20260709 12:14:57.834301 25168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 1019) took 1723 ms (client timeout 2066 ms). Trace:
I20260709 12:14:57.758230 28131 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:57.838279 25168 rpcz_store.cc:269] 0709 12:14:56.111070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.111363 (+   293us) service_pool.cc:224] Handling call
0709 12:14:57.834106 (+1722743us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54001e8054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:57.834229 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702092032}
W20260709 12:14:57.838181 25169 rpcz_store.cc:269] 0709 12:14:56.030151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.097497 (+ 67346us) service_pool.cc:224] Handling call
0709 12:14:57.830397 (+1732900us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400880254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:57.830497 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379665152}
I20260709 12:14:57.842484 23825 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:14:57.841938 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:57.843492 28049 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.845680 28049 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:57.854063 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:57.855010 23927 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:57.856115 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:57.856596 23749 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:57.857019 23749 leader_election.cc:403] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:14:57.858242 28157 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:57.824246 28334 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:57.805593 28233 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:14:57.827514 28238 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:57.745810 28266 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:57.868010 25176 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:57.874667 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.824579 28093 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:14:57.859166 28157 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:14:57.886963 28157 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:57.887496 28157 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:57.888782 28157 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.890614 28157 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:57.863966 28247 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:57.863447 28298 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:57.892989 28298 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:57.892477 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:57.894274 28298 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:57.892470 28247 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:57.859022 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:57.796869 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:57.862831 28327 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
W20260709 12:14:57.900348 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 103561us
W20260709 12:14:57.862675 28302 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.004s
I20260709 12:14:57.994283 25168 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:57.860409 27949 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:58.003209 28045 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:14:58.047582 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.897091 28247 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:57.891310 28131 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:58.004529 28177 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:58.009614 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:58.061295 23779 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:58.033612 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.000341 28240 consensus_meta.cc:287] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.907s	user 0.009s	sys 0.006s
I20260709 12:14:58.014207 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:14:57.902096 28300 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.946s	user 0.002s	sys 0.008s
I20260709 12:14:57.996133 28233 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.088788 28325 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.859549 28260 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:58.094044 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 79948us
I20260709 12:14:58.099578 28131 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:58.105275 28045 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:58.106151 28045 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:58.111974 28166 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.112792 28166 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.062582 28298 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:58.006511 28238 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:58.117672 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3198) took 1173 ms. Trace:
I20260709 12:14:57.947682 28093 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:58.018694 28302 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:58.120289 28325 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.148802 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2692) took 1948 ms. Trace:
I20260709 12:14:58.148897 23856 rpcz_store.cc:276] 0709 12:14:56.944514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.945037 (+   523us) service_pool.cc:224] Handling call
0709 12:14:56.945341 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.557639 (+612298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.117352 (+559713us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54002df354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.117596 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378778880}
I20260709 12:14:58.163784 23782 rpcz_store.cc:276] 0709 12:14:56.200698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.201113 (+   415us) service_pool.cc:224] Handling call
0709 12:14:56.201476 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:56.236557 (+ 35081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.148485 (+1911928us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54000bd454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.148717 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26709888}
I20260709 12:14:58.107503 28045 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.096832 28247 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:58.153467 28163 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.173107 28163 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.173410 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:58.091250 28240 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.174175 23854 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:58.051914 28289 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:14:58.119037 25175 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:58.090958 27949 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:58.154248 28333 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.045792 28322 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:58.164443 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:58.154424 28166 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.097325 28233 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.068423 28177 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.189632 28177 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.190946 28322 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.173904 28163 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.194706 28163 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.197269 28233 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.180100 23927 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:58.179368 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:58.199476 23851 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:14:58.125674 28327 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:58.198082 23856 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:58.202647 28244 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.040s	sys 0.071s
I20260709 12:14:58.177397 28238 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:58.186512 27999 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.181056 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:58.213487 23856 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:14:58.194571 28322 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.223067 23779 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
W20260709 12:14:58.211977 28244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.040s	sys 0.071s
W20260709 12:14:58.234031 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.955s	user 0.041s	sys 0.049s
W20260709 12:14:58.234432 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.041s	sys 0.049s
I20260709 12:14:58.234539 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:58.243002 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 969365us
I20260709 12:14:58.243872 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:14:58.260767 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:58.275324 23779 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:14:58.276859 27950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.022s	sys 0.038s
I20260709 12:14:58.284137 28131 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:14:58.285362 27950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.023s	sys 0.039s
I20260709 12:14:58.286115 28131 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:58.283560 28303 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:58.286057 27950 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:14:58.304335 28303 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:58.305437 28303 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.308266 28290 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:58.315927 27949 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.277773 28289 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:58.210986 28327 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:58.356510 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.278002 28302 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:58.366215 28314 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:58.372939 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.374006 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.373515 28238 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:58.393131 23781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2709) took 2114 ms (client timeout 1575 ms). Trace:
W20260709 12:14:58.393610 23781 rpcz_store.cc:269] 0709 12:14:56.268506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.268685 (+   179us) service_pool.cc:224] Handling call
0709 12:14:58.382683 (+2113998us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b540010a754. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:58.382847 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70563072}
I20260709 12:14:58.394963 28157 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.396813 27974 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:58.397768 27974 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:58.398631 28050 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:58.399241 28050 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:14:58.400620 28050 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.402557 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2769) took 1140 ms. Trace:
I20260709 12:14:58.414539 23926 rpcz_store.cc:276] 0709 12:14:57.262260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.262380 (+   120us) service_pool.cc:224] Handling call
0709 12:14:57.315932 (+ 53552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.367433 (+ 51501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.402259 (+1034826us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b5400334ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.402485 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113564928}
I20260709 12:14:58.404170 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2773) took 1083 ms. Trace:
W20260709 12:14:58.405881 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.417317 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.406491 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:58.407260 23780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2758) took 1811 ms (client timeout 2115 ms). Trace:
I20260709 12:14:58.361989 28290 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.412137 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:58.422873 23856 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:14:58.358414 28240 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.420188 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 629) took 1054 ms. Trace:
I20260709 12:14:58.423826 23852 rpcz_store.cc:276] 0709 12:14:57.366211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.371063 (+  4852us) service_pool.cc:224] Handling call
0709 12:14:57.385003 (+ 13940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.385383 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.385413 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.394597 (+  9184us) log.cc:844] Serialized 26 byte log entry
0709 12:14:57.599855 (+205258us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:57.599893 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.599957 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.405718 (+805761us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:14:58.411237 (+  5519us) raft_consensus.cc:1685] finished
0709 12:14:58.411261 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.420102 (+  8841us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":371,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":27711,"spinlock_wait_cycles":9634048,"thread_start_us":166651,"threads_started":1,"wal-append.queue_time_us":866828}
I20260709 12:14:58.415956 23925 rpcz_store.cc:276] 0709 12:14:57.320306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.320427 (+   121us) service_pool.cc:224] Handling call
0709 12:14:57.320730 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.587248 (+266518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.403913 (+816665us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002b6954. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.404110 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611460480}
I20260709 12:14:58.426574 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:58.418594 23780 rpcz_store.cc:269] 0709 12:14:56.608072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.642127 (+ 34055us) service_pool.cc:224] Handling call
0709 12:14:58.407065 (+1764938us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400370ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:58.407192 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554970368}
W20260709 12:14:58.428966 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.429176 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:58.429759 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.429782 23858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1381) took 1677 ms (client timeout 1914 ms). Trace:
I20260709 12:14:58.429895 23925 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:14:58.430433 23858 rpcz_store.cc:269] 0709 12:14:56.752595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.753113 (+   518us) service_pool.cc:224] Handling call
0709 12:14:58.429585 (+1676472us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54002df854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:58.429720 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42028288}
I20260709 12:14:58.431802 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.433135 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.433837 23925 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:58.440678 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.441107 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:58.444248 28346 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:14:58.435261 28261 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.446213 28164 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.446930 28164 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.447992 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.449575 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:58.449803 28164 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.450558 23926 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:14:58.450899 28327 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.452881 23926 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:58.470312 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:58.474149 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1273) took 2010 ms. Trace:
I20260709 12:14:58.474634 23929 rpcz_store.cc:276] 0709 12:14:56.463301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.610603 (+147302us) service_pool.cc:224] Handling call
0709 12:14:56.611217 (+   614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.719181 (+107964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:56.719227 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:56.719273 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:56.719311 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.473835 (+1754524us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400203ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.473894 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.474072 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247782144}
W20260709 12:14:58.480401 23785 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2756) took 1885 ms (client timeout 1851 ms). Trace:
W20260709 12:14:58.480922 23785 rpcz_store.cc:269] 0709 12:14:56.595051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.624468 (+ 29417us) service_pool.cc:224] Handling call
0709 12:14:58.480137 (+1855669us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b540010c554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:58.480321 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1687061888}
I20260709 12:14:58.487709 28280 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:58.485970 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.490664 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:58.491460 23854 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
W20260709 12:14:58.435070 28334 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.565s	user 0.001s	sys 0.014s
I20260709 12:14:58.509837 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:58.510803 23781 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:58.511718 23779 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:58.515120 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:58.516338 23779 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:58.516373 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:58.517786 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:58.516577 23781 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:14:58.518924 23780 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:58.524137 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:58.500241 23852 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:14:58.528496 23785 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:14:58.537546 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:14:58.531141 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2687) took 2333 ms. Trace:
I20260709 12:14:58.507303 28050 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:58.547017 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.431191 28290 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.420063 28157 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.540380 23777 rpcz_store.cc:276] 0709 12:14:56.198194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.198835 (+   641us) service_pool.cc:224] Handling call
0709 12:14:56.199144 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:56.246035 (+ 46891us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:56.488255 (+242220us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.687322 (+199067us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:56.687858 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:14:56.687994 (+   136us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:57.594606 (+906612us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b1800148600. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf 00007fb2ad7cc038
0709 12:14:57.594810 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 12:14:57.595058 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.530815 (+935757us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b54003a4654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.530877 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.530928 (+    51us) raft_consensus.cc:1685] finished
0709 12:14:58.530940 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.531081 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":895749,"prepare.queue_time_us":1377624,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3987,"raft.queue_time_us":7015,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":4970,"spinlock_wait_cycles":105851520,"thread_start_us":362427,"threads_started":2,"wal-append.queue_time_us":228323}
I20260709 12:14:58.511480 28327 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:58.512626 28334 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:58.527254 28302 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:58.563905 23855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.022s	sys 0.087s
I20260709 12:14:58.564167 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:14:58.582206 23855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.053s	user 0.022s	sys 0.088s
I20260709 12:14:58.598527 27989 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.553001 27853 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:14:58.599831 27853 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:58.600214 27853 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:14:58.601331 27853 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.610383 27955 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.621127 28332 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:58.621857 28332 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:58.625301 28290 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:58.642732 28266 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.007s
I20260709 12:14:58.622965 27974 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.696702 23680 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 168) took 11034 ms. Trace:
I20260709 12:14:58.698388 27974 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.603965 28302 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.587271 28327 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.664661 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2768) took 1403 ms. Trace:
I20260709 12:14:58.704998 23934 rpcz_store.cc:276] 0709 12:14:57.261674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.261799 (+   125us) service_pool.cc:224] Handling call
0709 12:14:57.262103 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.590309 (+328206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.664336 (+1074027us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54002b7d54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.664581 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753462656}
I20260709 12:14:58.668042 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1283) took 1003 ms. Trace:
I20260709 12:14:58.706355 23927 rpcz_store.cc:276] 0709 12:14:57.665121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.882939 (+217818us) service_pool.cc:224] Handling call
0709 12:14:57.883268 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:58.198760 (+315492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.667762 (+469002us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54002b7354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:58.667970 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682111360}
I20260709 12:14:58.679010 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:58.687636 28260 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:58.688632 28304 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.708407 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:58.707996 28304 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:58.692087 23776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2707) took 2452 ms (client timeout 2238 ms). Trace:
W20260709 12:14:58.711915 23776 rpcz_store.cc:269] 0709 12:14:56.239451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.239577 (+   126us) service_pool.cc:224] Handling call
0709 12:14:58.691859 (+2452282us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400167e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:58.692001 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150649600}
I20260709 12:14:58.694967 28325 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:58.713032 28304 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.715929 28327 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.586495 27999 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:58.697265 23680 rpcz_store.cc:276] 0709 12:14:47.662308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.668039 (+  5731us) service_pool.cc:224] Handling call
0709 12:14:47.828213 (+160174us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b5400002490. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68db71 00007fb2aa68e0b2 00007fb2b3607759 00007fb2b359fa76 00007fb2b3570d58 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc
0709 12:14:58.696567 (+10868354us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:53.089238 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:53.089861 (+   623us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:53.089920 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:14:53.089941 (+    21us) tablet.cc:662] Decoding operations
0709 12:14:53.091011 (+  1070us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:53.091136 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:53.091166 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:14:53.155606 (+ 64440us) tablet.cc:701] Row locks acquired
0709 12:14:53.198867 (+ 43261us) write_op.cc:260] PREPARE: finished
0709 12:14:53.259217 (+ 60350us) write_op.cc:270] Start()
0709 12:14:53.387400 (+128183us) write_op.cc:276] Timestamp: P: 1783599293212667 usec, L: 0
0709 12:14:53.402276 (+ 14876us) op_driver.cc:348] REPLICATION: starting
0709 12:14:53.534597 (+132321us) log.cc:844] Serialized 3954 byte log entry
0709 12:14:54.233779 (+699182us) op_driver.cc:464] REPLICATION: finished
0709 12:14:54.629864 (+396085us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18002acf40. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:14:54.631223 (+  1359us) write_op.cc:301] APPLY: starting
0709 12:14:54.634852 (+  3629us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:54.822458 (+187606us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:54.822487 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:55.405176 (+582689us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:55.405440 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:14:55.405491 (+    51us) write_op.cc:312] APPLY: finished
0709 12:14:55.405828 (+   337us) log.cc:844] Serialized 97 byte log entry
0709 12:14:55.554249 (+148421us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:55.554406 (+   157us) write_op.cc:454] Released schema lock
0709 12:14:55.559295 (+  4889us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:55.603324 (+ 44029us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:55.605855 (+  2531us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2106300,"reported_tablets":14,"spinlock_wait_cycles":8020352,"child_traces":[["op",{"apply.queue_time_us":150159,"apply.run_cpu_time_us":31279,"apply.run_wall_time_us":964514,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":168687,"prepare.run_cpu_time_us":32870,"prepare.run_wall_time_us":959998,"raft.queue_time_us":18467,"raft.run_cpu_time_us":50788,"raft.run_wall_time_us":168669,"replication_time_us":824177,"spinlock_wait_cycles":10425728,"thread_start_us":789083,"threads_started":5,"wal-append.queue_time_us":698259,"wal-append.run_cpu_time_us":9419,"wal-append.run_wall_time_us":2313611}]]}
I20260709 12:14:58.623216 28332 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.718778 28332 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:58.720075 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.611209 27955 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.750011 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.749104 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.766635 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.750726 23929 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
W20260709 12:14:58.783582 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.788138 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:58.741338 27853 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:58.786803 28049 consensus_meta.cc:287] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.932s	user 0.009s	sys 0.005s
I20260709 12:14:58.740658 28304 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.791230 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:58.801330 28157 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.804414 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:14:58.806797 28298 consensus_meta.cc:287] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.002s
I20260709 12:14:58.807538 23929 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:58.807974 23929 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:14:58.807490 25119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.005s	sys 0.010s
I20260709 12:14:58.809067 23929 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:58.811811 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.810417 25119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.006s	sys 0.010s
I20260709 12:14:58.771025 28334 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:14:58.811787 23929 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:58.809767 28289 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:58.808143 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50982.7us
I20260709 12:14:58.810943 28304 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.822211 28334 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:14:58.813710 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 657845us
I20260709 12:14:58.845340 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:58.858346 23933 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.361s	user 0.033s	sys 0.012s
W20260709 12:14:58.861778 23933 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.364s	user 0.033s	sys 0.012s
I20260709 12:14:58.862015 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:58.863131 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:58.863693 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:14:58.866011 28349 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:58.867185 28233 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:58.867583 28233 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:14:58.868618 28233 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:58.869975 28233 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:58.866075 28306 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:58.823295 28232 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:14:58.877832 27955 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:58.848428 28281 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.878760 28281 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.867444 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:58.883364 23854 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:58.863878 23852 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:58.866017 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:14:58.922245 23860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.105s	user 0.020s	sys 0.053s
W20260709 12:14:58.924628 23860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.020s	sys 0.053s
I20260709 12:14:58.842694 28334 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:14:58.859902 28247 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.005s
W20260709 12:14:58.923991 23931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.030s	sys 0.045s
W20260709 12:14:58.932839 23931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.030s	sys 0.045s
W20260709 12:14:58.933945 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 70352.4us
I20260709 12:14:58.881270 28281 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:58.944857 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.944978 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:14:58.932356 28354 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 12:14:58.925225 23679 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 4 to 8, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 8 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:58.947237 28318 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.124s	user 0.013s	sys 0.026s
W20260709 12:14:58.938326 28353 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.122s	user 0.001s	sys 0.002s
I20260709 12:14:58.958596 23854 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:14:58.961889 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:14:58.956677 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:58.977490 28093 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
W20260709 12:14:59.008049 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51448.8us
W20260709 12:14:58.933583 28264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.142s	user 0.030s	sys 0.045s
I20260709 12:14:58.949717 28305 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:58.977788 28134 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:59.011559 28264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.032s	sys 0.049s
W20260709 12:14:59.006492 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.948091 28232 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:59.010341 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48528.5us
I20260709 12:14:59.011161 28281 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:59.037540 28281 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:59.037972 28281 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:59.037386 28232 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:59.042706 28232 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:59.043825 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.048998 28281 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.052011 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:59.052829 23858 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:59.053272 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:59.053936 23854 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:59.034487 28305 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:58.995416 23679 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 5 to 9, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 9 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.066308 28290 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.067727 23778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.407s	user 0.048s	sys 0.057s
W20260709 12:14:59.068115 23778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.048s	sys 0.057s
W20260709 12:14:59.005291 28318 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.013s	sys 0.027s
W20260709 12:14:59.079295 23783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.000s	sys 0.024s
W20260709 12:14:59.079807 23783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.000s	sys 0.025s
I20260709 12:14:59.062012 23679 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 8 to 10, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 10 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.115335 23854 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
W20260709 12:14:59.138477 23857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.286s	user 0.014s	sys 0.036s
W20260709 12:14:59.180377 23928 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.506s	user 0.016s	sys 0.041s
W20260709 12:14:59.148998 25169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.251s	user 0.000s	sys 0.032s
W20260709 12:14:59.104144 25112 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.009s	sys 0.021s
I20260709 12:14:59.012332 28044 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.068416 28134 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:14:59.479744 27949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.120s	user 0.003s	sys 0.023s
W20260709 12:14:59.480329 27949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.003s	sys 0.023s
I20260709 12:14:59.175624 28157 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:59.172703 28161 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.034s	sys 0.060s
W20260709 12:14:59.483131 28161 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.034s	sys 0.060s
W20260709 12:14:59.219172 28238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.017s	sys 0.022s
W20260709 12:14:59.484146 28238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.018s	sys 0.022s
W20260709 12:14:59.211057 28258 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.016s	sys 0.025s
W20260709 12:14:59.485219 28258 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.016s	sys 0.026s
I20260709 12:14:59.486474 28258 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.211628 28263 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.009s	sys 0.014s
I20260709 12:14:59.186688 28266 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.092181 28281 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:14:59.082399 28282 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.556s	user 0.013s	sys 0.028s
I20260709 12:14:59.067034 28290 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.108338 28294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.256s	user 0.028s	sys 0.057s
I20260709 12:14:59.086289 28298 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:59.143112 28304 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.011391 28306 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:59.159385 28307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.754s	user 0.010s	sys 0.028s
I20260709 12:14:59.200311 28305 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:58.996896 28334 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:14:59.104775 28335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.860s	user 0.008s	sys 0.032s
W20260709 12:14:59.187135 28355 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:14:59.284730 28362 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:14:59.331172 23749 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.010s	sys 0.000s
W20260709 12:14:59.330956 23748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.000s	sys 0.003s
W20260709 12:14:59.415802 25112 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.280s	user 0.009s	sys 0.022s
W20260709 12:14:59.331923 23750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.003s	sys 0.025s
W20260709 12:14:59.331382 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.007s	sys 0.004s
W20260709 12:14:59.401968 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.011s	sys 0.011s
W20260709 12:14:59.377633 23857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.525s	user 0.014s	sys 0.036s
W20260709 12:14:59.402438 23928 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.728s	user 0.016s	sys 0.041s
W20260709 12:14:59.415179 25169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.517s	user 0.000s	sys 0.033s
W20260709 12:14:59.376086 23851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.008s	sys 0.029s
I20260709 12:14:59.451939 28333 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.452915 28045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.065s	user 0.013s	sys 0.012s
W20260709 12:14:59.221522 28198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.001s	sys 0.023s
W20260709 12:14:59.459817 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.333593 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2744) took 1325 ms. Trace:
W20260709 12:14:59.332244 23784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.565s	user 0.008s	sys 0.026s
W20260709 12:14:59.376086 23859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.251s	user 0.020s	sys 0.028s
I20260709 12:14:59.365971 23679 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 3 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 8 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:59.217875 28303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.011s	sys 0.028s
I20260709 12:14:59.481846 28134 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.541129 23749 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.010s	sys 0.006s
W20260709 12:14:59.546929 23750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.094s	user 0.007s	sys 0.025s
W20260709 12:14:59.547318 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.129s	user 0.009s	sys 0.006s
W20260709 12:14:59.551232 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.013s	sys 0.013s
W20260709 12:14:59.552398 23748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.003s	sys 0.003s
I20260709 12:14:59.221961 28289 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:59.558802 23859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.433s	user 0.020s	sys 0.028s
W20260709 12:14:59.555473 23851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.329s	user 0.017s	sys 0.043s
W20260709 12:14:59.559314 23784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.792s	user 0.009s	sys 0.026s
W20260709 12:14:59.551071 28045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.168s	user 0.013s	sys 0.013s
I20260709 12:14:59.487519 28258 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.487706 28157 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.599185 25113 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.380s	user 0.011s	sys 0.018s
I20260709 12:14:59.606916 27949 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:14:59.613036 28335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.316s	user 0.011s	sys 0.035s
W20260709 12:14:59.612812 28294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.709s	user 0.031s	sys 0.057s
W20260709 12:14:59.607731 23852 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.021s
W20260709 12:14:59.602216 28302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.009s	sys 0.005s
W20260709 12:14:59.595778 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.04499e+06us
I20260709 12:14:59.619369 28388 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.619998 23933 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.019s	sys 0.044s
W20260709 12:14:59.617527 28198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.001s	sys 0.024s
W20260709 12:14:59.638031 28360 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.402s	user 0.001s	sys 0.001s
I20260709 12:14:59.625015 28334 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:59.626623 28307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.158s	user 0.012s	sys 0.031s
W20260709 12:14:59.627664 28366 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:14:59.630786 27999 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.654151 27999 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.630872 28381 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:14:59.617259 28263 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.146s	user 0.011s	sys 0.016s
W20260709 12:14:59.631994 28303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.251s	user 0.011s	sys 0.028s
I20260709 12:14:59.658950 28303 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.613689 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 131762us
W20260709 12:14:59.641728 23933 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.019s	sys 0.044s
W20260709 12:14:59.641897 28280 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.011s	sys 0.019s
W20260709 12:14:59.662127 28280 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.011s	sys 0.019s
W20260709 12:14:59.635365 25113 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.575s	user 0.012s	sys 0.018s
W20260709 12:14:59.616936 28282 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.032s	user 0.017s	sys 0.028s
I20260709 12:14:59.543751 23783 rpcz_store.cc:276] 0709 12:14:58.007796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.007957 (+   161us) service_pool.cc:224] Handling call
0709 12:14:58.008289 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:58.048500 (+ 40211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:58.048534 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:58.059971 (+ 11437us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.332601 (+1272630us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:59.332715 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.332852 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.332947 (+    95us) raft_consensus.cc:1685] finished
0709 12:14:59.332973 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.333309 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39774,"raft.queue_time_us":41721,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":10355,"thread_start_us":1271815,"threads_started":1,"wal-append.queue_time_us":1173196}
W20260709 12:14:59.640724 28302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.010s	sys 0.006s
I20260709 12:14:59.662155 28381 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.664284 23852 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.013s	sys 0.023s
I20260709 12:14:59.672959 28370 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.674295 23679 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 8 to 9, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 9 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.675448 28266 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.676191 28362 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
W20260709 12:14:59.676942 28361 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.456s	user 0.001s	sys 0.001s
I20260709 12:14:59.619167 28298 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.680197 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2757) took 3325 ms. Trace:
W20260709 12:14:59.680910 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 1.62091e+06us
I20260709 12:14:59.632982 27963 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:59.685315 28258 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.686477 28198 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:59.688179 28384 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.689985 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3219) took 1920 ms. Trace:
I20260709 12:14:59.690366 28385 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.690793 28309 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:59.693636 23855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 630) took 2172 ms. Trace:
I20260709 12:14:59.676867 28388 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.615691 28333 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.669338 27949 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.613446 28044 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.697021 25169 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 1037) took 1801 ms. Trace:
I20260709 12:14:59.697955 28303 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.695097 28281 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.659741 28306 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.657225 28290 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:59.707198 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.657244 27999 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.683390 28157 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.709656 23928 rpcz_store.cc:276] 0709 12:14:56.354483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.356528 (+  2045us) service_pool.cc:224] Handling call
0709 12:14:56.356830 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:56.397953 (+ 41123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:56.397985 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.673985 (+1276000us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b2c000bb9c8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ac0aba11 00007fb2ac0bb372 00007fb2ad7eda01 00007fb2ad7b243a 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2
0709 12:14:57.674406 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.614394 (+1939988us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:59.632186 (+ 17792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.632358 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.679666 (+ 47308us) raft_consensus.cc:1685] finished
0709 12:14:59.679698 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.680035 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66828,"raft.queue_time_us":42386,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":12359,"spinlock_wait_cycles":20383616,"thread_start_us":1860361,"threads_started":1,"wal-append.queue_time_us":1574064}
W20260709 12:14:59.726738 23825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.028s
I20260709 12:14:59.726538 23679 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 2 to 5, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 5 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:59.728960 28233 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.817s	user 0.004s	sys 0.016s
W20260709 12:14:59.729772 28232 consensus_meta.cc:287] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.014s
I20260709 12:14:59.732753 28381 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.737288 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1280) took 2116 ms. Trace:
W20260709 12:14:59.738251 27853 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.944s	user 0.015s	sys 0.007s
I20260709 12:14:59.741632 28298 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:59.756310 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.745818 25172 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
W20260709 12:14:59.756893 23826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.018s	sys 0.042s
W20260709 12:14:59.763165 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.000s	sys 0.058s
W20260709 12:14:59.744673 23860 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.029s	sys 0.028s
I20260709 12:14:59.739413 28266 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:14:59.768309 23825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.008s	sys 0.028s
I20260709 12:14:59.770887 25176 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:59.774534 23826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.019s	sys 0.043s
I20260709 12:14:59.755798 28370 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.776801 28381 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.777853 28381 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.779170 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2731) took 2507 ms. Trace:
I20260709 12:14:59.776332 28281 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.779750 23778 rpcz_store.cc:276] 0709 12:14:57.271264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.620122 (+348858us) service_pool.cc:224] Handling call
0709 12:14:57.620450 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.620789 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.620817 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.659856 (+ 39039us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.068488 (+1408632us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:59.068531 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.068608 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.765210 (+696602us) raft_consensus.cc:1685] finished
0709 12:14:59.768075 (+  2865us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.778928 (+ 10853us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2860,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":6643,"thread_start_us":1408145,"threads_started":1,"wal-append.queue_time_us":1561595}
I20260709 12:14:59.782042 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 404) took 3053 ms. Trace:
I20260709 12:14:59.782622 28281 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.776923 28306 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.784597 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:59.788264 25175 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:14:59.785598 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:59.790582 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2776) took 2320 ms. Trace:
I20260709 12:14:59.791059 23933 rpcz_store.cc:276] 0709 12:14:57.470613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.470755 (+   142us) service_pool.cc:224] Handling call
0709 12:14:57.474371 (+  3616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.474591 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:58.862247 (+1387656us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.746452 (+884205us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.761358 (+ 14906us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:59.763707 (+  2349us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:59.773300 (+  9593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.790335 (+ 17035us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.790513 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47812,"prepare.queue_time_us":51133,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":72885,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":18277,"spinlock_wait_cycles":8207360,"thread_start_us":2163940,"threads_started":2,"wal-append.queue_time_us":828828}
I20260709 12:14:59.794510 28044 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:59.795570 23932 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1275) took 2730 ms (client timeout 1893 ms). Trace:
W20260709 12:14:59.796098 23932 rpcz_store.cc:269] 0709 12:14:57.065552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.065717 (+   165us) service_pool.cc:224] Handling call
0709 12:14:59.795158 (+2729441us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b54002026d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:14:59.795329 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5951246592}
I20260709 12:14:59.784716 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:59.773454 28384 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.799903 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:59.801070 28282 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:59.801682 28282 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:59.770057 23931 rpcz_store.cc:276] 0709 12:14:57.621020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.622034 (+  1014us) service_pool.cc:224] Handling call
0709 12:14:57.622334 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.622855 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.622894 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.629112 (+  6218us) log.cc:844] Serialized 26 byte log entry
0709 12:14:58.933325 (+1304213us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:58.933373 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.933446 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.736790 (+803344us) raft_consensus.cc:1685] finished
0709 12:14:59.736835 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.737097 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2515,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":10373,"thread_start_us":1198888,"threads_started":1,"wal-append.queue_time_us":1535287}
I20260709 12:14:59.804076 28384 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.804735 28384 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.805049 28374 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:59.805467 28374 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:14:59.805984 28374 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:59.806350 23932 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:59.808621 27963 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:59.809130 25177 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:59.811043 28318 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.810840 28384 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.811869 28388 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.811740 28318 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.812469 28388 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.813112 28388 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:14:59.803311 28378 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.184s	user 0.000s	sys 0.003s
I20260709 12:14:59.759577 25169 rpcz_store.cc:276] 0709 12:14:57.895762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.895927 (+   165us) service_pool.cc:224] Handling call
0709 12:14:57.896335 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:57.896749 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.896780 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:57.897490 (+   710us) log.cc:844] Serialized 52 byte log entry
0709 12:14:59.610887 (+1713397us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:59.617726 (+  6839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.648113 (+ 30387us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.660523 (+ 12410us) raft_consensus.cc:1685] finished
0709 12:14:59.660540 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.696934 (+ 36394us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3086,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":84126,"thread_start_us":1637299,"threads_started":1,"wal-append.queue_time_us":1347561}
I20260709 12:14:59.752085 23855 rpcz_store.cc:276] 0709 12:14:57.520541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.520735 (+   194us) service_pool.cc:224] Handling call
0709 12:14:57.522460 (+  1725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.522822 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.522852 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.527801 (+  4949us) log.cc:844] Serialized 26 byte log entry
0709 12:14:58.582803 (+1055002us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:58.582848 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.582922 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.692859 (+1109937us) raft_consensus.cc:1685] finished
0709 12:14:59.692916 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.693255 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":663,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"thread_start_us":1053569,"threads_started":1,"wal-append.queue_time_us":1463444}
I20260709 12:14:59.782450 25172 rpcz_store.cc:276] 0709 12:14:56.728772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.729028 (+   256us) service_pool.cc:224] Handling call
0709 12:14:56.749434 (+ 20406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:59.779756 (+3030322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.781636 (+  1880us) spinlock_profiling.cc:243] Waited 2.95 s on lock 0x7b54005ea1d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:14:59.781948 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6777092864}
W20260709 12:14:59.828509 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.000s	sys 0.059s
I20260709 12:14:59.830945 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3226) took 1678 ms. Trace:
I20260709 12:14:59.831441 23851 rpcz_store.cc:276] 0709 12:14:58.152701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.200143 (+ 47442us) service_pool.cc:224] Handling call
0709 12:14:58.200534 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:58.201410 (+   876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:58.201444 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:58.204294 (+  2850us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.647774 (+1443480us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:59.657795 (+ 10021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.657920 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.802785 (+144865us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b24003d8a88. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa54b9c4 00007fb2aa549acb 00007fb2aa704dc9 00007fb2aa704a55 00007fb2aa70409d 00007fb2ad33610e 00007fb2ad34d88c 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221
0709 12:14:59.830534 (+ 27749us) raft_consensus.cc:1685] finished
0709 12:14:59.830556 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.830870 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":307,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5384,"spinlock_wait_cycles":30894848,"thread_start_us":1343726,"threads_started":1,"wal-append.queue_time_us":1402483}
I20260709 12:14:59.832866 25175 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:59.814172 28381 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:59.806461 28399 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
W20260709 12:14:59.805557 28376 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.073s	user 0.001s	sys 0.002s
W20260709 12:14:59.835661 23860 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.029s	sys 0.030s
I20260709 12:14:59.801164 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:59.724324 23859 rpcz_store.cc:276] 0709 12:14:57.769925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.772879 (+  2954us) service_pool.cc:224] Handling call
0709 12:14:57.773154 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.773353 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:57.798232 (+ 24879us) log.cc:844] Serialized 14 byte log entry
0709 12:14:58.125250 (+327018us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b180036ac60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:14:59.633629 (+1508379us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:59.662926 (+ 29297us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:59.689407 (+ 26481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.689545 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.689854 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24713,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":52405,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":287,"spinlock_wait_cycles":45266176,"thread_start_us":1440643,"threads_started":1,"wal-append.queue_time_us":1889721}
I20260709 12:14:59.756639 28198 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.809211 27963 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:59.844494 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.813174 28384 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.745406 28385 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.797585 28333 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:14:59.814173 28318 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:14:59.851743 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 50733.7us
I20260709 12:14:59.862090 28385 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.858904 28381 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.863312 28381 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.866845 28381 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.868631 28388 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.803222 28282 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.872102 28405 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:59.873119 28306 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.873775 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.874130 28404 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:14:59.874881 28404 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.854893 27999 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:59.877357 28404 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.878464 28298 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.879211 28298 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.880620 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:59.881217 27853 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.881357 23851 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:14:59.881860 27853 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.872864 28405 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.859365 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3217) took 2090 ms. Trace:
I20260709 12:14:59.881762 28298 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.891045 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:59.877591 27999 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:14:59.912149 23781 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:59.912595 23781 pending_rounds.cc:85] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 8
I20260709 12:14:59.912850 23781 pending_rounds.cc:107] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:14:59.913282 27853 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.913093 27999 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.913655 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:59.878573 28309 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.915171 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:14:59.915970 23779 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:59.918063 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:59.913101 23781 raft_consensus.cc:2889] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:59.908241 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:59.919983 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2739) took 2643 ms. Trace:
I20260709 12:14:59.909230 25119 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:14:59.909502 23929 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:59.910331 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:59.921594 28372 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:59.874902 28370 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.922336 23782 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:14:59.922921 27949 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:14:59.923380 28318 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:59.923869 28318 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.924247 28318 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:59.924403 28303 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.915447 28309 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.928305 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:14:59.890559 28381 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.929591 28381 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.930099 23858 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:59.897593 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:59.918213 28198 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.920382 23784 rpcz_store.cc:276] 0709 12:14:57.276348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.627046 (+350698us) service_pool.cc:224] Handling call
0709 12:14:57.627375 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:57.634599 (+  7224us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:57.733096 (+ 98497us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.628154 (+1895058us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:59.629585 (+  1431us) log.cc:844] Serialized 52 byte log entry
0709 12:14:59.629855 (+   270us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:59.629897 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.630039 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.919716 (+289677us) raft_consensus.cc:1685] finished
0709 12:14:59.919733 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.919914 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15225,"prepare.queue_time_us":73298,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":68641,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":7490,"thread_start_us":1798598,"threads_started":1,"wal-append.queue_time_us":1498698}
I20260709 12:14:59.914696 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:59.899847 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.929023 23854 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:59.929046 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:59.932852 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3218) took 2163 ms. Trace:
I20260709 12:14:59.886294 28044 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.815930 28232 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:14:59.886281 28385 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:59.979223 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.985212 28402 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:14:59.986056 28402 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.986894 28405 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.987627 28044 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:14:59.989459 27853 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.886691 23857 rpcz_store.cc:276] 0709 12:14:57.769355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.769475 (+   120us) service_pool.cc:224] Handling call
0709 12:14:57.769800 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.770114 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.770139 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.784596 (+ 14457us) log.cc:844] Serialized 26 byte log entry
0709 12:14:59.617220 (+1832624us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:59.761281 (+144061us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.775084 (+ 13803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.838143 (+ 63059us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.838184 (+    41us) raft_consensus.cc:1685] finished
0709 12:14:59.838196 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.859296 (+ 21100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10070,"prepare.queue_time_us":29557,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":52,"raft.queue_time_us":17272,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":12781,"thread_start_us":1811902,"threads_started":2,"wal-append.queue_time_us":1404444}
W20260709 12:14:59.924434 28332 consensus_meta.cc:287] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.205s	user 0.003s	sys 0.020s
I20260709 12:14:59.912163 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.977605 23779 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:00.017190 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.927546 28309 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.931564 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:14:59.979956 28404 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:00.014439 23778 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:15:00.015187 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.996711 28402 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:14:59.891100 28266 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.980140 28298 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:14:59.996762 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.170948 23784 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:59.995515 27949 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.995863 27853 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.015077 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.022447 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:14:59.995527 28044 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.032133 28118 consensus_queue.cc:1059] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:59.997124 28305 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.010761 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:00.012521 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.020861 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.031162 28261 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:00.039455 28232 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.016237 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.008219 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:14:59.992205 28157 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.007563 28398 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.012151 28304 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.018353 28289 raft_consensus.cc:3055] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:00.017767 23851 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 7.
I20260709 12:15:00.014777 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.028630 28307 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:00.087569 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.194314 23933 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:00.027750 28332 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:14:59.981995 25114 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.508s (SENT) [suppressed 20 similar messages]
I20260709 12:14:59.948148 23860 rpcz_store.cc:276] 0709 12:14:57.769650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.772803 (+  3153us) service_pool.cc:224] Handling call
0709 12:14:57.815878 (+ 43075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.816199 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.816229 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.816809 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:14:58.926952 (+1110143us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:59.856027 (+929075us) log.cc:844] Serialized 14 byte log entry
0709 12:14:59.932448 (+ 76421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.932565 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:59.932634 (+    69us) raft_consensus.cc:1685] finished
0709 12:14:59.932650 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:59.932793 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102,"prepare.queue_time_us":727933,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":80,"raft.queue_time_us":11895,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":8066,"thread_start_us":2005235,"threads_started":2,"wal-append.queue_time_us":1386705}
I20260709 12:15:00.007581 28385 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.001196 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.032820 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.006731 27963 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.008042 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:14:59.933110 28403 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.000164 28233 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:00.027806 28381 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.031937 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.207712 23781 raft_consensus.cc:2376] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10. Re-sending same reply.
I20260709 12:15:00.207636 28398 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.020532 23929 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:00.207955 28304 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:00.191071 28289 raft_consensus.cc:740] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:00.211099 28403 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.212023 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.205497 28402 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.213274 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.213483 28388 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.214546 28403 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.214250 28388 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:00.216571 27963 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:00.216737 28388 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.202905 28263 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.218786 28263 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:00.171967 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 113925us
W20260709 12:15:00.170188 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 287102us
I20260709 12:15:00.050307 28405 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.180454 28298 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.182085 27853 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.182860 25172 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:00.183478 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.210409 28304 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:00.183837 28261 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.184460 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.184744 28266 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.185019 28232 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.182868 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:00.183898 23857 raft_consensus.cc:2410] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:15:00.188836 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.189127 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 642) took 1178 ms. Trace:
I20260709 12:15:00.190668 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:00.203153 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 221177us
I20260709 12:15:00.199868 23854 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:00.205781 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:00.210348 28398 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.020382 28290 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.163094 23778 pending_rounds.cc:85] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 7
I20260709 12:15:00.246590 23778 pending_rounds.cc:107] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:15:00.246865 23778 raft_consensus.cc:2889] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:00.231674 23851 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:00.234843 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.184217 28407 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.237798 23779 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:15:00.238894 25119 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:00.194797 28408 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.218817 28385 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.198151 28157 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.199091 23853 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:00.245751 23852 rpcz_store.cc:276] 0709 12:14:59.010276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.010518 (+   242us) service_pool.cc:224] Handling call
0709 12:14:59.045515 (+ 34997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.045729 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:59.755475 (+709746us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.174115 (+418640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.179285 (+  5170us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:00.188831 (+  9546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.188908 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.189056 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619840,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":93,"raft.queue_time_us":5194,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5513,"thread_start_us":1036895,"threads_started":2}
I20260709 12:15:00.248081 23929 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:15:00.248739 23925 raft_consensus.cc:3060] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:00.249682 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:00.251171 23854 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:00.252019 28232 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.253087 23778 pending_rounds.cc:107] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:15:00.254424 28261 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.261420 28266 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:00.262885 25113 connection.cc:256] RPC call timeout handler was delayed by -1.6206025209999098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
I20260709 12:15:00.264032 28289 consensus_queue.cc:260] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.268263 23784 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:15:00.298156 23859 raft_consensus.cc:2376] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13. Re-sending same reply.
I20260709 12:15:00.274510 28298 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.275484 23783 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:15:00.289674 23860 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:00.244812 28388 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.210989 28307 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:00.293438 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54639.8us
I20260709 12:15:00.236486 28381 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.300349 23778 raft_consensus.cc:2889] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:00.411612 25172 raft_consensus.cc:1240] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:00.345618 25113 connection.cc:256] RPC call timeout handler was delayed by -1.5273867079999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20179999999999998s for a response
W20260709 12:15:00.300081 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.211341 28332 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.206666 27949 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.195602 28305 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:00.414729 25113 connection.cc:256] RPC call timeout handler was delayed by -1.4492619760001162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26230000000000003s for a response
I20260709 12:15:00.390286 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:00.300246 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 65954.7us
I20260709 12:15:00.341018 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:00.364929 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:00.356204 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.342161 28289 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:00.429378 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 179827us
I20260709 12:15:00.430780 28332 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:00.431962 23749 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:00.420462 28408 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.436415 23779 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:00.438211 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:00.300333 28281 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:00.439227 23851 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:00.350052 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:00.239609 28402 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.442785 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.341275 28370 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.459375 28164 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.460220 28164 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.462738 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.464054 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.459453 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.464054 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.466136 23776 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:00.470937 28263 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.471825 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:00.471849 23932 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:00.474677 27949 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.453526 23934 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.475740 28232 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:00.323863 28290 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.320075 28405 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.478435 28290 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.325731 28404 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.480674 28307 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.481382 28404 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.435923 28385 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.482976 23927 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:00.490849 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.493654 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:00.493884 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 326343us
W20260709 12:15:00.484014 28334 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.785s	user 0.008s	sys 0.004s
I20260709 12:15:00.450423 28281 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:00.494326 23783 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
W20260709 12:15:00.501120 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 70977.9us
I20260709 12:15:00.479216 28405 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.483647 28294 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:00.492128 28374 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.003s
I20260709 12:15:00.486137 28385 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.514612 23925 raft_consensus.cc:2468] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:15:00.517892 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.350682 28407 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.518615 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:00.526121 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.426934 28381 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.505762 28281 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:00.506549 28290 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:00.554767 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:00.518092 28385 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.564074 23785 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:00.548357 28405 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:00.529057 28157 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:00.573042 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145213us
W20260709 12:15:00.576930 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 275113us
I20260709 12:15:00.586083 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.584370 25170 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:00.522367 28374 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:00.592501 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44134.6us
I20260709 12:15:00.577004 27999 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.530071 28294 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.600286 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.587263 28232 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.605130 25176 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:00.565423 28388 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.589186 28408 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.586311 28305 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.611110 28290 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.616562 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.613317 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:00.617697 25168 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:00.618115 23782 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:00.619534 23898 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:00.616235 28157 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.621639 28157 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.622525 28164 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:00.613528 28318 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.688s	user 0.003s	sys 0.009s
W20260709 12:15:00.624269 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 306778us
W20260709 12:15:00.625895 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116713us
I20260709 12:15:00.617660 28232 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.619946 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.634163 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.625192 25173 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:00.630874 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:00.618331 28416 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:15:00.639681 23858 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:00.634042 28232 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:00.628657 28335 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:00.642186 23785 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.644346 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:00.628119 28157 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.649555 28388 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.649894 23857 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:00.644061 28335 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.640843 28398 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.661348 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.664193 28407 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.647481 28332 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:00.665540 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45765.5us
I20260709 12:15:00.659694 28417 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:00.655408 28424 log.cc:927] Time spent T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:15:00.672086 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.671025 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:00.638317 28318 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.673820 23782 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:00.684553 23785 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:00.684890 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:00.692024 28388 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.692759 28388 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.692799 28398 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.693665 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.678551 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.691919 27999 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:00.696343 28407 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:00.696758 28398 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:00.676250 28402 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.695238 28388 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.670399 28335 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:00.707379 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.708083 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.708977 23780 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.709901 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:00.711238 28307 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:00.695941 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54275.4us
I20260709 12:15:00.697567 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.700366 28134 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:00.697520 27999 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.702526 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.675868 28417 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.670902 28431 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:15:00.717654 23856 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.673224 28332 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:00.705837 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.706660 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.720906 23778 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.707131 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.696041 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.708024 23779 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.713690 25169 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:00.711552 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:00.698156 28374 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:00.719333 23785 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:00.721632 23783 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:15:00.718797 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 272214us
I20260709 12:15:00.724388 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.725124 23783 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:00.717871 28318 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.748121 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.748464 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.750389 28402 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:00.750984 28402 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:00.723255 28332 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.753003 28381 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:00.746134 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47701.2us
I20260709 12:15:00.752007 28402 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.748203 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:00.744515 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:00.756538 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45093.2us
I20260709 12:15:00.757925 23782 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:15:00.750651 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:00.718226 27999 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.768034 28307 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.768487 28307 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.768824 28307 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.772504 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:00.773695 25114 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:00.774724 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:00.766830 28431 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:00.775157 28318 raft_consensus.cc:2804] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:00.775444 28431 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:00.775429 28398 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:00.776111 28431 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.767941 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.774747 28134 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:00.777787 28134 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:00.778149 28134 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:00.780226 28282 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.780556 28118 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:00.782712 28233 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:00.784034 28381 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:00.785084 28407 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:00.785555 28407 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:00.785775 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 89845.1us
I20260709 12:15:00.786566 28407 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.787096 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.787894 28398 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:00.788256 28407 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.788842 28430 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.789590 28430 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:00.791399 23858 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:15:00.791944 28374 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:00.793349 28281 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.794466 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.809648 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:00.795518 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.801359 23779 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:00.785279 28318 raft_consensus.cc:697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:00.803745 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.816669 23778 raft_consensus.cc:1813] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Handling vote request from an unknown peer 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.804922 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.817051 28318 consensus_queue.cc:237] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.818671 28289 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:00.819470 28266 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.817169 23778 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:00.808378 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.821120 23785 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:00.799418 23856 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:00.804333 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.776309 28164 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:00.813167 28282 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:00.815834 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.831871 23779 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:00.817572 23780 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:00.787278 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.822733 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:00.834888 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.718281 28417 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:00.819777 28289 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:00.830407 23778 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:00.840060 25177 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:00.835644 23783 raft_consensus.cc:2376] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10. Re-sending same reply.
I20260709 12:15:00.826013 28164 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.843273 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.823524 28374 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:15:00.843914 28164 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:00.846273 28374 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.847196 28304 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.820087 28266 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.849543 28266 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.830505 28370 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:00.850293 28370 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:00.850728 28370 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:00.852634 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.839748 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:00.843079 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 56079.5us
I20260709 12:15:00.856104 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.868492 23900 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:00.868767 28414 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.869933 28294 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:00.870355 28045 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:00.871224 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:00.871976 28045 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:00.872133 28281 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.872918 28281 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:00.873370 28281 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:00.873155 28045 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.874298 28281 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.876305 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.880131 23785 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:00.881145 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.881819 23785 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:00.882654 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.883679 23785 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:00.884553 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.882884 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:00.887809 23785 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:00.889372 23785 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:15:00.889353 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51628.2us
W20260709 12:15:00.889966 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 165012us
I20260709 12:15:00.891566 23751 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:00.892609 28414 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:00.895754 27949 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:00.896601 28385 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:00.893147 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:00.839412 28289 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:00.919687 23856 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:00.922520 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:00.928120 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.928920 23751 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9.
I20260709 12:15:00.938266 23779 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:00.939463 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.941852 23781 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:00.945654 23854 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:00.979271 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:00.979900 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 77447.3us
I20260709 12:15:00.981043 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.981760 23779 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:00.986863 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.987272 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.987905 23856 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:15:00.991053 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:00.889683 28335 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:00.993368 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:00.995970 23852 raft_consensus.cc:2410] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:00.998018 28385 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:00.998829 28420 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:00.912428 28134 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.005193 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.009333 28304 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:00.991792 23853 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:01.015825 28304 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:01.015108 23779 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:15:01.021867 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 199214us
I20260709 12:15:01.018112 28304 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:01.024241 28304 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.025563 28304 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.029650 25169 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:01.007233 28370 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.022024 23854 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:01.030210 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.014079 28420 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:01.035144 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.021588 28414 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:01.039934 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 235718us
I20260709 12:15:01.037943 23852 pending_rounds.cc:85] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 6
I20260709 12:15:01.041397 23852 pending_rounds.cc:107] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:15:01.041739 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.045882 25168 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:01.041669 23852 raft_consensus.cc:2889] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:01.047771 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.013098 28289 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.061692 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.034049 28307 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.077821 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.069037 28430 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:01.042518 28431 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.089005 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.074122 28289 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.092103 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.092904 23925 raft_consensus.cc:2393] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 9: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
W20260709 12:15:01.131112 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.067404 28332 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:01.171635 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 130016us
I20260709 12:15:01.080292 28420 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.093950 28385 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.180006 23934 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:01.182614 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.189200 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.090580 28164 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.193362 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.195003 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.187083 28420 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:01.197008 23925 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:01.195712 23927 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:01.203953 23782 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
W20260709 12:15:01.205554 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 162629us
I20260709 12:15:01.192348 28335 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:01.193744 28134 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:01.236958 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.237301 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.238104 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.238554 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.239326 23934 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:01.239452 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.239627 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.240077 23931 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:01.190331 28385 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.204094 28370 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:01.242374 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.242911 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.231127 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.244043 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.247160 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.254165 23927 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:15:01.251456 23926 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:01.223246 28445 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:01.261082 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:01.261940 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 256193us
W20260709 12:15:01.266431 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 92333.1us
I20260709 12:15:01.265713 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.265534 23932 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:01.272732 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.273545 23934 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:01.280332 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.283396 23899 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.272186 28307 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:01.336455 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61783.3us
I20260709 12:15:01.277223 28370 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:15:01.338757 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 66835us
I20260709 12:15:01.339797 23899 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.348174 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:01.352329 28442 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:15:01.278118 28306 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:15:01.271602 28398 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:15:01.352149 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.360435 23851 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:01.366227 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.358749 28335 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.371941 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:01.367015 28439 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:15:01.362659 28334 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:01.381302 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.391074 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:01.359249 28462 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:01.393138 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.396040 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.396793 23780 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:01.397982 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:01.405642 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.410046 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.408095 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:01.413877 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.413954 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.415302 23932 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:01.415339 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.379908 28045 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:01.417898 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.419333 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:01.420671 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94421.2us
I20260709 12:15:01.423062 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.424458 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:01.421111 23934 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:01.399398 28334 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.389775 28402 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:01.431167 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.431552 28402 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:01.432118 28402 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.432848 28407 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:01.433264 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:01.428349 27999 consensus_meta.cc:287] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.003s
I20260709 12:15:01.434099 27999 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.435583 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.436232 27999 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:01.436761 28440 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:15:01.437657 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.438484 28266 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:01.438948 28266 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.439455 28266 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:01.440634 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.441958 25176 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:01.438514 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.444366 23857 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:01.387912 28164 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:01.445850 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.446218 28458 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:01.446456 23860 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:01.446789 28458 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.447432 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.448662 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.449160 28458 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:01.449863 27949 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.450323 27949 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.450693 27949 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.451851 23824 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.449266 23933 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:01.395620 28462 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.436487 23927 raft_consensus.cc:2410] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:15:01.407302 28431 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:01.460448 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.431079 28334 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:01.457427 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 73016.2us
W20260709 12:15:01.439126 28456 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:01.474828 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.431104 28233 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:01.465540 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.410470 28453 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.484071 25168 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:01.484658 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:01.485321 23855 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.486246 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.457906 28045 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.472254 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.467725 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.477926 28473 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:01.479156 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.479388 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.507851 23858 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:01.480087 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.480435 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.480988 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.482487 28477 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:01.482715 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.492436 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.470896 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.517614 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.500093 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.517875 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.456593 28462 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.517971 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.504462 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.519928 25170 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:01.433662 28407 consensus_queue.cc:1243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 15.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:15:01.449885 28332 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:01.457954 28443 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 12:15:01.462913 28467 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:15:01.475404 25171 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:01.451997 28414 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.484036 28453 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:01.414026 28451 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:15:01.509140 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54643us
I20260709 12:15:01.512364 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.509150 25169 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:01.514008 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.548799 23779 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:01.514652 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:01.514752 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48031.8us
I20260709 12:15:01.520939 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.549813 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:01.515985 25177 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:01.513788 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.526152 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.527251 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.550755 23780 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:01.532889 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.551505 23782 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:01.542088 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.515782 28477 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.522480 28385 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:01.554402 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:01.569844 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 107994us
I20260709 12:15:01.550756 23930 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:01.559262 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.525360 28475 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.571905 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.573019 25175 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:15:01.573405 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 59476.2us
I20260709 12:15:01.570211 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.574651 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.531610 28407 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:15:01.574633 23899 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.575894 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.542652 28414 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.580022 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.581905 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.583844 23824 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.583005 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:01.569964 28304 consensus_meta.cc:287] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.004s
I20260709 12:15:01.523027 28045 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.548158 28453 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.589782 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.588846 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.594359 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.570416 28305 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:01.591642 25176 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:01.601034 23785 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:01.570676 28477 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.596572 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.571926 28475 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.585095 28463 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:01.584789 28290 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:01.618634 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:01.592242 28417 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.002s
I20260709 12:15:01.589093 28332 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:01.619442 23933 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:01.627012 23779 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:01.597960 28385 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:01.604638 28304 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:01.627128 28472 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:15:01.654278 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.584638 28452 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 12:15:01.630489 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 46741.2us
W20260709 12:15:01.636910 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.638490 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.644588 23777 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:01.669062 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.649080 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.649960 25176 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:01.651970 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:01.665863 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.670544 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.670857 25175 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.671803 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:01.672605 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.628075 28407 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:01.673312 28407 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:01.669349 23777 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:01.648229 28455 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:15:01.670373 23856 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:01.630158 28475 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.679991 23855 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:01.680827 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.623633 28481 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:01.678821 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:01.685384 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88878us
I20260709 12:15:01.687355 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:01.689684 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.693272 25176 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:01.640244 28385 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.693270 23825 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:01.690052 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.694125 23853 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:01.636780 28332 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:01.699357 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.700309 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:01.702478 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.700033 23853 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:01.707419 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.707898 23776 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:01.708304 23779 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:01.699096 28332 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:01.709291 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.674260 28306 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:01.636402 28417 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.712255 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.681607 28290 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.712646 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.680517 28473 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:01.691784 28481 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.719157 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.720199 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.694783 28305 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:01.723201 23900 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.700260 28385 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.726364 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.726419 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.727939 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.729355 28478 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:01.693655 28487 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:01.713102 28459 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.730609 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:01.743239 23776 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:01.730080 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57548.6us
I20260709 12:15:01.743898 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.707093 28118 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:01.743940 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.718428 28290 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.748701 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:01.694674 28479 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:15:01.752377 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.748095 28459 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.753436 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:01.723024 28481 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:01.753896 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60727.2us
I20260709 12:15:01.708621 27949 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.719003 28294 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:15:01.730188 28233 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:01.723670 28419 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:15:01.759665 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:01.770201 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:01.750643 28290 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.717088 28306 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.765766 28488 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:01.775102 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:01.775835 28469 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:15:01.795753 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64861.7us
I20260709 12:15:01.754479 28118 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.776439 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:01.757774 23853 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:01.780102 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.799829 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.771497 28459 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.780453 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:01.762620 25175 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:01.714675 28407 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.752372 28452 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:01.728183 28473 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.751477 28281 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:15:01.805182 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:01.817680 23853 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.805199 25176 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:01.780404 28305 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:01.818428 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48329.7us
I20260709 12:15:01.825845 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:01.791056 28463 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:01.748225 28480 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
W20260709 12:15:01.794274 28474 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:15:01.836638 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.839967 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.798713 28488 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:01.807466 28451 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:01.811254 28294 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:01.813355 28045 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:01.803817 28304 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.838707 23926 raft_consensus.cc:2376] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9. Re-sending same reply.
I20260709 12:15:01.815052 28452 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.813220 28233 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:01.875063 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:01.872294 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 128438us
I20260709 12:15:01.884413 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.792104 28334 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:01.825390 28305 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.829625 28463 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:01.845932 28451 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:01.883364 23898 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:01.889565 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 89816.5us
I20260709 12:15:01.911541 23853 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:01.916004 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:01.869987 28294 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:01.918155 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:01.845081 28488 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:01.920604 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94866.2us
I20260709 12:15:01.921460 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:01.841149 28307 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:15:01.925164 23851 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:01.925869 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:01.874887 28452 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:01.928269 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:01.873525 28304 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:01.902405 28305 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.914494 28463 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.913543 28407 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:01.961243 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47289.9us
W20260709 12:15:01.963698 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 80388.1us
W20260709 12:15:01.964241 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 163346us
I20260709 12:15:01.917181 28451 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:01.922900 28318 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:01.983197 23780 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:01.987035 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:01.989023 25172 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.989197 23782 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:15:01.975095 28487 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:01.991369 28487 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:01.991832 28487 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.997429 25172 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
W20260709 12:15:01.998128 28476 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:15:01.998454 28496 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:01.998853 28118 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.054605 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:01.985222 28233 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.002517 28487 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:15:02.029364 28289 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.934s	user 0.012s	sys 0.000s
I20260709 12:15:01.970101 28417 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.008773 27949 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.026132 23776 raft_consensus.cc:1275] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:15:02.019467 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 135133us
I20260709 12:15:02.040467 23931 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
W20260709 12:15:02.041409 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69545.6us
W20260709 12:15:02.043169 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 121799us
I20260709 12:15:01.999198 28486 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:02.032514 28500 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.060160 28492 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:15:02.063041 28441 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.967s	user 0.006s	sys 0.000s
I20260709 12:15:02.070494 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:02.020193 28489 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:15:01.972687 28485 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:15:02.023030 28266 consensus_meta.cc:287] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.001s
W20260709 12:15:02.007115 28335 consensus_meta.cc:287] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.000s
I20260709 12:15:02.076653 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:15:02.082901 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 166953us
I20260709 12:15:02.088796 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:02.086061 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 203108us
I20260709 12:15:02.026449 28306 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.056145 28496 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.090593 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.033195 28232 consensus_queue.cc:1059] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:15:02.108659 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.090508 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.072993 28487 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:01.999126 28304 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.106258 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:02.106153 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.103988 28485 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.117583 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.087826 28486 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:02.114820 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.075924 28477 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.082553 28281 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:02.103520 28045 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.071516 28451 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.137791 23929 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:02.125814 28266 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.145955 28266 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.121953 28496 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.151027 23858 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:02.120322 27999 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:02.125442 28335 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:02.130292 28318 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:02.129698 28492 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:02.162636 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76732.4us
I20260709 12:15:02.133785 28486 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:02.094949 28500 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:02.157896 28402 consensus_meta.cc:287] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.001s
I20260709 12:15:02.152402 28306 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:15:02.157912 28414 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
W20260709 12:15:02.182684 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 76617.3us
I20260709 12:15:02.074333 28294 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.168788 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.177826 23932 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:02.127269 28485 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:02.186546 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.161190 28473 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.188227 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2773) took 1383 ms. Trace:
I20260709 12:15:02.182358 23777 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.135481 28332 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.188654 23784 rpcz_store.cc:276] 0709 12:15:00.804919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.805032 (+   113us) service_pool.cc:224] Handling call
0709 12:15:00.805339 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.805766 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.805802 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:00.806316 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:15:01.028197 (+221881us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:01.028242 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.028313 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:02.062143 (+1033830us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:02.182878 (+120735us) raft_consensus.cc:1685] finished
0709 12:15:02.182902 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.188144 (+  5242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2520,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":92,"spinlock_wait_cycles":172218112,"thread_start_us":221460,"threads_started":1,"wal-append.queue_time_us":256498}
I20260709 12:15:02.173682 27999 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:02.198808 23934 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 12:15:02.178855 28306 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:02.202500 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.184973 28318 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:02.192824 28462 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.661s	user 0.005s	sys 0.007s
W20260709 12:15:02.200559 28453 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.003s
I20260709 12:15:02.194787 28370 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:02.183049 28402 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.223989 28281 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.224203 28509 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:02.225696 28492 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.185561 28414 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.185231 28045 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:02.264506 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.218827 28499 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.254106 28513 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.275025 28334 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.216924 28290 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.275717 28516 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:02.279040 23785 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:02.232079 28370 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:02.280776 28499 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.298106 28499 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.297863 28513 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.224025 28502 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:02.252741 28509 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:02.288296 23931 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.208892 28462 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:02.292994 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 73043.8us
I20260709 12:15:02.221118 28453 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.311342 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:02.326117 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.326840 23860 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:02.286922 28481 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.329099 23854 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:02.300794 28502 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:02.332139 23925 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:02.309217 28509 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.318706 28517 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:02.337440 23931 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:02.347255 28134 consensus_queue.cc:1048] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:15:02.348135 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.314240 28516 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:02.359076 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.365780 28305 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.371127 23933 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:02.280675 27999 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.402029 28281 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:02.420375 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 152519us
I20260709 12:15:02.422997 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.418277 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.425971 28502 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.427217 28502 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.298395 28370 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.356305 28517 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:02.401182 28508 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.440397 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:02.455796 23930 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:02.444530 28045 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:02.475080 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 290943us
I20260709 12:15:02.318179 28318 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.453143 28332 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.480321 28481 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.480952 28500 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.434598 28281 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.457147 28517 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:02.472963 28508 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.499392 28508 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.500454 28539 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.476713 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:02.502925 28407 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.001s
I20260709 12:15:02.497385 28528 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.503729 23778 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:15:02.504820 28485 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.505757 28407 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:02.501516 28539 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:02.506916 28385 consensus_meta.cc:287] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.000s
I20260709 12:15:02.507128 28485 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.508255 28453 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.521904 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.526818 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:02.496076 28488 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 12:15:02.541776 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:02.480333 28477 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:02.512130 28463 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.004s
I20260709 12:15:02.545835 28305 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:02.547878 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 107575us
W20260709 12:15:02.539788 28475 consensus_meta.cc:287] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.000s
I20260709 12:15:02.507395 28385 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.515635 28485 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:02.538777 28459 consensus_meta.cc:287] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.720s	user 0.009s	sys 0.002s
I20260709 12:15:02.545658 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.543725 28508 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:02.555683 23900 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.541728 28539 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.552091 28462 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.606112 23779 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:02.554262 28414 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.555377 28290 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.555346 28532 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:15:02.628367 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 441847us
I20260709 12:15:02.547540 28488 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.602054 28481 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.549664 28463 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.701344 28463 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.557416 28459 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.704943 28459 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.706169 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "102493ff11494b009f79963bb57be511"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:15:02.630543 28525 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.071s	user 0.003s	sys 0.000s
I20260709 12:15:02.557091 28485 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.552651 28475 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.627545 28539 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:02.706983 25177 raft_consensus.cc:606] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:02.705133 28451 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:02.709219 28481 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.539330 28516 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.633159 28513 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.693496 28407 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:02.743223 23781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.012s
W20260709 12:15:02.754030 23781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.005s	sys 0.013s
I20260709 12:15:02.754729 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2781) took 1371 ms. Trace:
I20260709 12:15:02.693149 28534 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:02.744678 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 200205us
I20260709 12:15:02.757408 23857 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:02.754972 23781 rpcz_store.cc:276] 0709 12:15:01.383544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.384229 (+   685us) service_pool.cc:224] Handling call
0709 12:15:01.405351 (+ 21122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.409339 (+  3988us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:01.428008 (+ 18669us) log.cc:844] Serialized 14 byte log entry
0709 12:15:02.154256 (+726248us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b1800193660. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:15:02.754413 (+600157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:02.754470 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:02.754517 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.754586 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.754679 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4191,"prepare.queue_time_us":18648,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":308230,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":48636160,"thread_start_us":595643,"threads_started":2,"wal-append.queue_time_us":823112}
W20260709 12:15:02.738982 28431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.006s	sys 0.052s
I20260709 12:15:02.751673 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:02.749607 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.762501 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:02.747057 28481 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.753188 28528 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 12:15:02.764660 28530 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:15:02.764987 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:02.747841 28281 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.757359 28478 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:02.780131 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 691368us
I20260709 12:15:02.768744 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:15:02.780530 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 665710us
I20260709 12:15:02.780184 28507 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:02.780735 28304 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.770202 28488 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.781947 28304 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.782714 28304 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.781486 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:02.786223 28503 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.786643 28503 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.786983 28503 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.788661 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 626) took 1084 ms. Trace:
I20260709 12:15:02.760977 28517 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.789605 28402 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.781474 28507 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.790248 28507 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:02.771159 28486 consensus_meta.cc:287] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.004s
I20260709 12:15:02.791903 28486 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.794328 28486 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.760344 28534 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.759086 28407 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.796128 28385 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.798540 28517 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:02.771687 28544 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:15:02.800339 28515 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.530s	user 0.003s	sys 0.000s
I20260709 12:15:02.802251 23776 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:02.803489 23751 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.806720 23925 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 622) took 1103 ms. Trace:
I20260709 12:15:02.781541 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:02.773890 28494 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:02.810060 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.812901 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:02.810290 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.759197 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.816236 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:02.816314 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57188.9us
I20260709 12:15:02.818488 25176 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:02.818869 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.807166 23925 rpcz_store.cc:276] 0709 12:15:01.703225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.083183 (+379958us) service_pool.cc:224] Handling call
0709 12:15:02.806480 (+723297us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400204ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:02.806636 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500657536}
I20260709 12:15:02.821070 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.820978 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:02.814143 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:02.780992 28536 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:15:02.841439 28481 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:02.781925 28538 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:15:02.853936 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.855202 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:02.742563 28370 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:15:02.826822 23855 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:02.828104 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:02.828233 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.831050 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:02.831611 28335 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:02.832216 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:02.826519 28517 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.865595 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:02.866945 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 110305us
I20260709 12:15:02.833631 28407 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.844928 28488 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.867970 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.819162 28478 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:02.811956 28494 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:02.834527 28513 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:02.835179 28534 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:02.844579 28495 log.cc:927] Time spent T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.802s	user 0.000s	sys 0.002s
I20260709 12:15:02.845005 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:02.845367 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:02.846021 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:02.825250 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.885576 23859 raft_consensus.cc:2410] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:15:02.846815 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:02.861811 28481 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:02.849359 28542 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:15:02.851665 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 100100us
I20260709 12:15:02.852152 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:02.856083 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:02.825901 23927 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:15:02.856125 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.789084 23929 rpcz_store.cc:276] 0709 12:15:01.704263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.154440 (+450177us) service_pool.cc:224] Handling call
0709 12:15:02.154793 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:02.155156 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:02.292496 (+137340us) log.cc:844] Serialized 14 byte log entry
0709 12:15:02.356155 (+ 63659us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:02.370274 (+ 14119us) spinlock_profiling.cc:243] Waited 8.97 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:02.402002 (+ 31728us) log.cc:844] Serialized 26 byte log entry
0709 12:15:02.402178 (+   176us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:02.402233 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.402306 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:02.788347 (+386041us) raft_consensus.cc:1685] finished
0709 12:15:02.788457 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.788593 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26082,"prepare.queue_time_us":181401,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":16664,"raft.queue_time_us":91536,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":43632768,"thread_start_us":150228,"threads_started":2,"wal-append.queue_time_us":66251}
I20260709 12:15:02.891464 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:02.860591 23853 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.860251 23856 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:02.819577 28431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.007s	sys 0.052s
I20260709 12:15:02.870501 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:02.826042 28385 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:02.851266 28281 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:15:02.902820 28551 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:02.903163 28281 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:02.903371 28490 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:02.886569 23776 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:02.903872 28490 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:02.904294 28490 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:02.904712 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:02.847883 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:02.864565 28539 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:02.907222 23781 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:02.886302 28266 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:02.909940 28294 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.913158 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:02.914166 28370 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:02.914629 28370 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:02.914999 28370 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:02.891393 23925 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:02.861526 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:02.892450 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.896191 23934 raft_consensus.cc:2410] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:15:02.871526 28478 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.877489 28494 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.902305 28549 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.917229 23933 raft_consensus.cc:2368] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
W20260709 12:15:02.917323 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55914.9us
I20260709 12:15:02.925733 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.929153 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:02.929862 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.926606 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:02.926788 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 113342us
I20260709 12:15:02.847339 28503 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.932361 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.932825 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 858) took 1420 ms. Trace:
I20260709 12:15:02.898874 28335 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:02.934999 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.938062 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:02.933666 23825 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.934846 23785 rpcz_store.cc:276] 0709 12:15:01.512557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.613928 (+101371us) service_pool.cc:224] Handling call
0709 12:15:01.634652 (+ 20724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:02.293655 (+659003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.932583 (+638928us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400166554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:02.932761 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471375104}
I20260709 12:15:02.875337 28488 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.903656 28281 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:02.946211 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:02.928735 28543 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:15:02.925429 28478 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:02.948931 23853 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:02.927302 28539 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:02.947927 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.955324 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.957978 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.949236 28519 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.956704 28318 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:15:02.971704 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2818) took 1529 ms. Trace:
I20260709 12:15:02.972347 28420 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:02.972806 28420 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:02.973405 28475 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:02.973613 28420 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.974898 28420 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:02.950326 28487 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.972499 28318 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:02.976244 28318 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:02.939445 28266 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:02.949409 28550 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:02.977955 28550 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:02.978363 28550 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:02.933401 28513 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:02.967787 28335 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.969702 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:02.970489 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1409) took 1628 ms (client timeout 1658 ms). Trace:
W20260709 12:15:02.992668 23851 rpcz_store.cc:269] 0709 12:15:01.341883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.392814 (+ 50931us) service_pool.cc:224] Handling call
0709 12:15:02.970268 (+1577454us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540037fd54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:02.970406 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219937152}
I20260709 12:15:02.964550 28549 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:02.946090 28517 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.979296 23751 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.972158 23928 rpcz_store.cc:276] 0709 12:15:01.436864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.681497 (+244633us) service_pool.cc:224] Handling call
0709 12:15:01.695781 (+ 14284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.696031 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.696060 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:01.713881 (+ 17821us) log.cc:844] Serialized 26 byte log entry
0709 12:15:01.817593 (+103712us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:01.817629 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.817703 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:02.843849 (+1026146us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:02.950351 (+106502us) raft_consensus.cc:1685] finished
0709 12:15:02.950375 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.966388 (+ 16013us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14268160,"thread_start_us":68709,"threads_started":1,"wal-append.queue_time_us":193492}
I20260709 12:15:02.980726 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:02.945786 28385 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:03.009722 28385 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.010211 28385 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.011126 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.012423 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:02.977090 28318 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.013476 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.013648 28318 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.965529 28407 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:03.014827 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.015252 28407 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.015913 28407 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.005720 28507 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.007273 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.020687 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.021960 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.022542 23928 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.007916 28532 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:03.028307 28496 consensus_meta.cc:287] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.881s	user 0.009s	sys 0.003s
W20260709 12:15:03.028961 28510 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.827s	user 0.000s	sys 0.003s
I20260709 12:15:03.028844 28496 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.031101 28546 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:03.031474 28496 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.032445 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2806) took 1322 ms. Trace:
I20260709 12:15:03.032830 23780 rpcz_store.cc:276] 0709 12:15:01.709702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.989163 (+279461us) service_pool.cc:224] Handling call
0709 12:15:01.989530 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.989744 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.989769 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:01.990297 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:15:02.080397 (+ 90100us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:15:02.080438 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.080505 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:03.028372 (+947867us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:03.032151 (+  3779us) raft_consensus.cc:1685] finished
0709 12:15:03.032172 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.032384 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8481024,"thread_start_us":89694,"threads_started":1,"wal-append.queue_time_us":172309}
I20260709 12:15:03.033715 28483 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.034617 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:03.034377 28483 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:02.996030 28513 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:03.008939 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45330.9us
I20260709 12:15:03.038070 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:03.036772 28513 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:02.983146 28539 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:03.004145 28485 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.043958 28487 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:03.044387 28499 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.746s	user 0.007s	sys 0.005s
I20260709 12:15:02.966986 28488 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.031616 28546 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:02.983011 28452 consensus_meta.cc:287] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.016s	user 0.007s	sys 0.005s
I20260709 12:15:03.046092 23780 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:03.052188 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1419) took 1707 ms. Trace:
I20260709 12:15:03.055084 23858 rpcz_store.cc:276] 0709 12:15:01.344860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.757316 (+412456us) service_pool.cc:224] Handling call
0709 12:15:01.773965 (+ 16649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:02.151580 (+377615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.051880 (+900300us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54000703d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:03.052107 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866609536}
I20260709 12:15:03.059043 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.060451 28281 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.061803 28281 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.044866 28499 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:03.037986 28509 consensus_meta.cc:287] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.002s
I20260709 12:15:03.074712 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.081226 28509 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.084810 28507 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.037367 28304 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.087383 28304 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.087930 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.088793 23783 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:03.089246 23855 raft_consensus.cc:2368] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:03.089672 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.090405 23783 raft_consensus.cc:2393] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 16: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:03.091174 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.067736 28452 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.092142 23931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1314) took 1047 ms. Trace:
I20260709 12:15:03.081670 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 860) took 1243 ms. Trace:
I20260709 12:15:03.097415 23778 rpcz_store.cc:276] 0709 12:15:01.837927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.055693 (+217766us) service_pool.cc:224] Handling call
0709 12:15:02.056021 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:02.504047 (+448026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.081406 (+577359us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400167454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:03.081604 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028749568}
I20260709 12:15:03.098641 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.099192 23778 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.093551 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.100379 23933 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.096909 23931 rpcz_store.cc:276] 0709 12:15:02.044426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.090168 (+ 45742us) service_pool.cc:224] Handling call
0709 12:15:03.091969 (+1001801us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000e4054. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.092080 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397895040}
I20260709 12:15:03.053071 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:03.102407 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49461.3us
I20260709 12:15:03.103253 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.109714 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.115270 28558 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:03.116199 28558 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.118854 28558 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:03.128017 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 181739us
I20260709 12:15:03.130282 23783 raft_consensus.cc:2368] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
W20260709 12:15:03.133503 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 81499.4us
I20260709 12:15:03.134807 23932 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:03.134814 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:03.136699 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:03.137872 23751 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.105052 28157 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:03.143347 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.145710 23855 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:03.097136 28520 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:03.149870 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.150728 23929 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:03.151013 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.151551 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.152282 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.152734 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.107546 28485 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:03.159478 28414 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:03.161422 28407 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.163709 28502 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:03.165099 28499 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.164791 28490 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.166368 28503 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.166996 28490 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.143190 28546 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.170346 28478 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.170656 28385 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.171501 23776 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:03.134490 28549 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:03.179215 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.182552 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2804) took 1473 ms. Trace:
I20260709 12:15:03.144508 28516 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:03.144695 28551 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:03.182968 23782 rpcz_store.cc:276] 0709 12:15:01.708847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.775908 (+ 67061us) service_pool.cc:224] Handling call
0709 12:15:01.988924 (+213016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.989712 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.182119 (+1192407us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54000dd4d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.182475 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489001728}
I20260709 12:15:03.201927 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.155499 28539 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.207903 23853 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:03.208290 28511 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.208743 28501 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.172744 28517 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:03.218369 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.221073 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:03.206998 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48946.1us
I20260709 12:15:03.225560 28483 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.226014 28452 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:03.226620 28494 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.180157 28509 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.230651 23782 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:03.239643 28431 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:03.244905 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.245808 23929 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:15:03.245026 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 92816.4us
I20260709 12:15:03.246800 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.247303 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:03.251891 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.252461 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:03.262435 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.264838 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.265092 28509 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:03.266966 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.267649 23782 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.269881 23751 leader_election.cc:400] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:03.271644 23751 leader_election.cc:403] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:03.265897 23929 raft_consensus.cc:2393] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:03.265920 28517 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:03.275152 28517 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:03.278009 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.278887 23855 raft_consensus.cc:2393] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:03.279117 23928 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:03.282048 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.282768 23860 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.175333 28520 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.262562 28551 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.291476 23929 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:03.302067 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.302199 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.266332 28549 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:03.275070 28490 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:03.323419 23778 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:03.228801 28501 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.244998 28511 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.310662 28559 raft_consensus.cc:993] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:03.332033 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.335911 23778 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:03.339958 28568 raft_consensus.cc:1081] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:03.339248 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.344585 23853 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:15:03.330664 28420 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.327382 28551 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.327910 28509 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:03.345263 23858 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:03.350946 28551 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.336474 28304 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.327555 28490 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.337356 28549 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:03.362507 23860 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:03.363632 23778 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:03.354314 28385 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.369045 28306 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.370223 28306 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.373378 23926 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:03.374248 28490 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:03.375123 23926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 629) took 1652 ms (client timeout 2192 ms). Trace:
W20260709 12:15:03.375528 23926 rpcz_store.cc:269] 0709 12:15:01.722419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.117207 (+394788us) service_pool.cc:224] Handling call
0709 12:15:03.374897 (+1257690us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54003516d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.375023 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168299904}
I20260709 12:15:03.376307 28520 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.343636 28370 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.378665 28370 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.382239 23782 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:03.383558 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.384380 23782 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:03.330314 28517 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:03.392702 28569 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:15:03.393420 28549 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.394409 28501 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.351987 28420 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.398530 23782 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:03.361469 28304 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:03.400902 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 176597us
I20260709 12:15:03.401882 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:03.403893 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.405457 23825 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.407092 23933 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:03.401826 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.408840 23926 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.409727 23825 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:03.411849 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.408293 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.411303 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:03.412484 23929 raft_consensus.cc:2376] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13. Re-sending same reply.
I20260709 12:15:03.412914 28488 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:03.413475 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.413936 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:03.413436 28488 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:03.414232 23929 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:03.415064 28488 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.416025 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.401544 28517 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:03.417333 28503 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:03.417989 28503 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:03.413872 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:03.418774 25169 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.419885 28420 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.420679 28304 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.423341 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.422616 28490 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.422262 28487 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:03.424166 28385 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.438982 23926 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:03.439699 25169 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:03.444269 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1316) took 1399 ms. Trace:
I20260709 12:15:03.444806 23932 rpcz_store.cc:276] 0709 12:15:02.045046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.195185 (+150139us) service_pool.cc:224] Handling call
0709 12:15:02.202089 (+  6904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.143669 (+941580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.443929 (+300260us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54002445d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:03.444181 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144598784}
I20260709 12:15:03.450299 28157 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.450919 28157 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.452195 28551 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.418967 28503 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.459044 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:03.460381 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.461522 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.462494 23782 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:15:03.425660 28511 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.422178 28550 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.480609 23927 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:03.466017 28385 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.492116 23899 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.493685 28479 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:03.494326 28479 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:03.495332 28479 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.497517 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.500809 28567 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.501549 28567 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.506654 28385 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.508083 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:03.517364 28570 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.177s	user 0.003s	sys 0.001s
I20260709 12:15:03.518353 28550 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:03.542362 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.526875 28507 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:03.542362 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.545042 23778 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:03.547433 23932 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
W20260709 12:15:03.558877 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:03.545436 28572 log.cc:927] Time spent T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:15:03.561367 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.563251 23782 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.551054 28567 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.571187 23782 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:03.546056 28478 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.572907 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.574851 28550 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:03.575359 28550 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:03.577049 28550 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.580670 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.581954 23782 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.581877 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.591733 23778 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.549918 28507 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.596843 23782 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:03.600312 23779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2802) took 1889 ms (client timeout 2011 ms). Trace:
W20260709 12:15:03.600960 23779 rpcz_store.cc:269] 0709 12:15:01.708352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.712442 (+  4090us) service_pool.cc:224] Handling call
0709 12:15:03.597609 (+1885167us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b540035b1d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.597776 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044985728}
W20260709 12:15:03.602563 23857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1416) took 2258 ms (client timeout 2239 ms). Trace:
W20260709 12:15:03.603057 23857 rpcz_store.cc:269] 0709 12:15:01.344077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.665513 (+321436us) service_pool.cc:224] Handling call
0709 12:15:03.602294 (+1936781us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400413cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.602449 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484666624}
I20260709 12:15:03.605146 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.600358 28494 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.613173 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.614482 23852 raft_consensus.cc:2368] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13.
I20260709 12:15:03.616135 28407 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.624631 25174 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:15:03.624797 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.625792 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.626461 23783 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.627303 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:03.628564 25173 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:03.632483 28509 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:03.636080 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:03.615466 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2815) took 1096 ms. Trace:
I20260709 12:15:03.638175 23781 rpcz_store.cc:276] 0709 12:15:02.518743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.811473 (+292730us) service_pool.cc:224] Handling call
0709 12:15:03.615254 (+803781us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b5400093154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.615361 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102252800}
I20260709 12:15:03.605850 23784 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.639735 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:03.573978 28503 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.637789 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:03.647606 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.650909 28487 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.653971 23858 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.653834 28487 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.654465 28318 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:03.654764 23855 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.626111 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.602870 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.657568 23779 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.658257 23749 leader_election.cc:400] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:03.658561 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.648886 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.659993 23860 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:03.650120 23930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 625) took 1946 ms (client timeout 1567 ms). Trace:
I20260709 12:15:03.648578 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.662179 23857 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.664563 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.665140 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.665884 23926 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:03.652354 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:03.656620 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.667196 23782 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
W20260709 12:15:03.660567 23930 rpcz_store.cc:269] 0709 12:15:01.704015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.776159 (+ 72144us) service_pool.cc:224] Handling call
0709 12:15:03.649944 (+1873785us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540032a854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.650043 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662162816}
I20260709 12:15:03.651662 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:03.619047 28157 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.672999 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:03.673743 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.659127 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.671934 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.678618 23851 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.679055 23859 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:03.679323 23779 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:03.678148 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.682312 23851 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:03.668256 28478 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:03.667979 28494 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.686528 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.687565 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.685626 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.641865 28490 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:03.688912 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.704005 23779 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.688403 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.711762 23851 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:03.691831 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:03.693557 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.696002 23784 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:03.657115 28534 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:03.723785 23934 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:03.722910 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.650964 28511 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.724581 23933 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:03.722831 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.697822 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.725971 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:03.692839 28484 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.059s	sys 0.053s
I20260709 12:15:03.671310 28532 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.734382 23851 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:03.734719 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.729089 25119 leader_election.cc:400] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13.
I20260709 12:15:03.732587 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.735664 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:03.668426 28304 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.737188 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:03.678083 28501 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.739933 23749 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.742671 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.745322 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:03.734285 28484 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.060s	sys 0.053s
I20260709 12:15:03.754070 23782 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:03.756474 23749 leader_election.cc:400] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:03.756847 23749 leader_election.cc:403] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:15:03.689459 28509 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:03.758376 23779 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:03.767082 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:03.723057 28539 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.768234 23782 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.779539 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2807) took 1262 ms. Trace:
I20260709 12:15:03.779912 23782 rpcz_store.cc:276] 0709 12:15:02.516708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.766793 (+1250085us) service_pool.cc:224] Handling call
0709 12:15:03.768062 (+  1269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.779275 (+ 11213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.779480 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:03.782537 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:03.768234 23781 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:03.769119 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.689281 28516 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:03.768213 23851 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:03.751426 28549 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:03.689880 28281 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.689592 28478 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:03.773615 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:03.670684 28503 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.777814 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:03.723809 28534 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:03.789168 28157 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.768723 23929 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:03.776692 28485 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:03.745966 28551 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.740702 28452 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.805316 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.731986 28045 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:15:03.751189 28134 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.819743 28385 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:03.816923 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 2811) took 1299 ms. Trace:
I20260709 12:15:03.774055 28532 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.820430 28385 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.820613 23776 rpcz_store.cc:276] 0709 12:15:02.517758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.904443 (+386685us) service_pool.cc:224] Handling call
0709 12:15:03.816715 (+912272us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003a4654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:03.816842 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612041600}
I20260709 12:15:03.822727 28385 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.819816 28452 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.825152 28452 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.827864 28549 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:03.828919 28549 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.829596 23777 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:03.832327 28487 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.766762 28559 consensus_queue.cc:1059] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:15:03.833812 27999 raft_consensus.cc:3055] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:03.834203 27999 raft_consensus.cc:740] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:03.834793 28507 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.835575 23785 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:15:03.838668 28546 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.744071 28494 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.840677 28485 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:03.842103 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.842526 28494 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.811527 28370 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.842931 23855 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:03.843441 28370 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:03.816252 28580 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:15:03.795652 28501 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:03.847740 23930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:03.848434 23930 raft_consensus.cc:606] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:03.848532 28306 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:03.849758 28306 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:03.762514 28511 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.805891 28516 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.850174 28306 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:03.850680 23852 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:03.808599 28520 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.844110 23858 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:03.852100 28520 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.853199 23858 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:03.854393 28520 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.855404 23855 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.820052 28134 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.815279 28431 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:03.856298 28431 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:03.856791 23855 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:03.774785 28304 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.841758 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:03.841751 28485 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:03.871693 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.872895 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:03.854619 23858 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
W20260709 12:15:03.808954 28577 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:15:03.875157 23852 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:15:03.876317 23783 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:03.894178 23778 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:03.894843 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 236325us
I20260709 12:15:03.899299 23784 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:03.895368 23858 pending_rounds.cc:85] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 9
I20260709 12:15:03.899730 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:03.900528 23858 pending_rounds.cc:107] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:15:03.900864 23858 raft_consensus.cc:2889] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:03.878317 23859 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:03.808250 28568 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.896369 23857 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:03.811333 28534 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.875236 23931 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:03.849829 28503 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.851109 28306 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:03.901960 28501 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.923703 28501 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.923969 28304 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.835114 27999 consensus_queue.cc:260] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.930955 23778 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:03.845983 28370 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.900722 23900 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:03.950115 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49624.7us
I20260709 12:15:03.950910 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:03.907631 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2812) took 1389 ms. Trace:
I20260709 12:15:03.906541 28157 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.938133 23780 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:03.953636 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2817) took 1434 ms. Trace:
I20260709 12:15:03.953985 23780 rpcz_store.cc:276] 0709 12:15:02.519221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.937526 (+1418305us) service_pool.cc:224] Handling call
0709 12:15:03.937929 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.953360 (+ 15431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.953571 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:03.857275 28431 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.944085 28306 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.894445 28524 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:15:03.946810 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 194948us
I20260709 12:15:03.908998 28385 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.966152 28385 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.968422 28385 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.906162 28511 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:03.974009 23777 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:15:03.974882 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 852) took 2465 ms. Trace:
I20260709 12:15:03.954948 23783 rpcz_store.cc:276] 0709 12:15:02.517999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.906758 (+1388759us) service_pool.cc:224] Handling call
0709 12:15:03.907081 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.907286 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:03.907314 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:03.907358 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:03.907388 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.907443 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:03.907550 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:03.976787 23783 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:03.977322 28134 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:03.977445 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2819) took 1036 ms. Trace:
W20260709 12:15:03.977999 28164 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:15:03.978467 28496 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.978967 28496 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.841675 28478 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:03.956095 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2816) took 1437 ms. Trace:
W20260709 12:15:03.959749 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 222641us
W20260709 12:15:03.945228 28578 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:15:03.907233 28550 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:15:03.904685 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.985227 23778 rpcz_store.cc:276] 0709 12:15:02.518982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.907990 (+1389008us) service_pool.cc:224] Handling call
0709 12:15:03.930773 (+ 22783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.955837 (+ 25064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.956038 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:03.977837 23783 rpcz_store.cc:276] 0709 12:15:02.941210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.976143 (+1034933us) service_pool.cc:224] Handling call
0709 12:15:03.976582 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:03.977214 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.977390 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:03.988042 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83428.3us
I20260709 12:15:03.962278 28493 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.975251 23777 rpcz_store.cc:276] 0709 12:15:01.509657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.510340 (+   683us) service_pool.cc:224] Handling call
0709 12:15:01.510708 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.644487 (+133779us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54003758a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad729dc0 00007fb2ad7b41c0 00007fb2ad7b5231 00007fb2ad7b1d8f 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c
0709 12:15:02.142481 (+497994us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4c001d7d80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aba7ea20 00007fb2aba8fa29 00007fb2aba55644 00007fb2aba7cf2d 00007fb2aba7cb93 00007fb2ad7ad18b 00007fb2ad7adcb5 00007fb2ad7a8f30 00007fb2ad7aa7de 00007fb2ad7b4b49 00007fb2ad7b5416
0709 12:15:03.829149 (+1686668us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b58007ee9d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad70e0aa 00007fb2ad790447 00007fb2ad7a8fe7 00007fb2ad7aa7de 00007fb2ad7b4b49 00007fb2ad7b5416 00007fb2ad7b1d8f 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4
0709 12:15:03.974570 (+145421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:03.974819 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3787750400}
I20260709 12:15:04.000480 25169 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:04.001152 23777 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:15:04.001833 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2828) took 1045 ms. Trace:
I20260709 12:15:03.998072 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.992015 27999 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.963665 28527 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:03.964629 28498 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:15:04.008491 28537 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:04.009930 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.010246 25177 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:04.010800 28534 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.012064 28417 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:04.012919 28417 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:04.015143 23783 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:04.015425 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.002393 23778 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.006209 28584 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:15:03.990093 28511 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:04.006985 28304 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:04.015923 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2827) took 1060 ms. Trace:
I20260709 12:15:04.022281 28304 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.022787 28304 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.006909 28452 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:03.980118 28045 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:15:03.999361 28550 consensus_queue.cc:1243] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:04.032655 28304 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.034482 23932 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:04.016974 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2821) took 1075 ms. Trace:
I20260709 12:15:04.033057 28534 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:04.035359 23778 rpcz_store.cc:276] 0709 12:15:02.941796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.986159 (+1044363us) service_pool.cc:224] Handling call
0709 12:15:04.002222 (+ 16063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.016464 (+ 14242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.016914 (+   450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.036885 23778 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:04.037314 28516 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.037632 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2830) took 1054 ms. Trace:
I20260709 12:15:04.038048 23778 rpcz_store.cc:276] 0709 12:15:02.983423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.036338 (+1052915us) service_pool.cc:224] Handling call
0709 12:15:04.036696 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.037364 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.037568 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.021768 23783 rpcz_store.cc:276] 0709 12:15:02.955920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.987338 (+1031418us) service_pool.cc:224] Handling call
0709 12:15:04.014893 (+ 27555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.015676 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.015869 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.002175 23777 rpcz_store.cc:276] 0709 12:15:02.956125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.000457 (+1044332us) service_pool.cc:224] Handling call
0709 12:15:04.000912 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:04.001578 (+   666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.001780 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.044000 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.044968 25169 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:04.047075 23898 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:03.980428 28492 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:15:04.054965 28550 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:04.056995 28493 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:04.059156 28527 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:04.060261 28527 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:04.060628 28527 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.061442 28527 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.062685 28527 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.066128 23781 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:15:04.069759 28045 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:04.070247 28045 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:04.070617 28045 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.071362 23857 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:04.072268 28582 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:15:04.075222 28516 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:04.075707 28516 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.076572 28516 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.072204 28483 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:04.079905 28483 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.080761 28483 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.081449 28452 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.081826 28524 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:04.083946 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:04.084797 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:04.084887 23781 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:04.085592 23857 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:04.088078 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:04.088914 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.087527 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:04.089323 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.019824 28544 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:04.082279 28524 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.021486 28511 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.116135 23855 raft_consensus.cc:2393] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:04.137966 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.039366 23778 raft_consensus.cc:1240] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:04.120122 23899 leader_election.cc:400] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
I20260709 12:15:04.124377 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:04.128193 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.129534 23857 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
W20260709 12:15:04.134387 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45259.7us
I20260709 12:15:04.129710 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.143392 23899 leader_election.cc:403] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:15:04.145267 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.153110 23856 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:04.152549 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.152845 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.170099 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.171687 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.168691 25174 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:04.173110 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.131713 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.081988 28483 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.132897 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:04.116925 28475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.004s	sys 0.017s
I20260709 12:15:04.184787 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.185405 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.186012 23930 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:04.186909 23856 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:15:04.187233 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 55652.8us
I20260709 12:15:04.188952 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.186965 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.190078 23933 raft_consensus.cc:2393] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:04.184787 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.190369 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.190923 23928 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:04.179147 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.177932 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:04.182127 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 869) took 1038 ms. Trace:
I20260709 12:15:04.192850 23778 rpcz_store.cc:276] 0709 12:15:03.129382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.038836 (+909454us) service_pool.cc:224] Handling call
0709 12:15:04.039172 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.144384 (+105212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.167483 (+ 23099us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:04.187482 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 444866us
I20260709 12:15:04.194976 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:04.115173 28544 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.196036 23857 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:04.082115 28452 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:15:04.197144 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:04.201254 23929 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:04.150132 28511 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:04.054019 28584 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.122s
I20260709 12:15:04.203541 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:04.214455 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.215641 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.174986 28516 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.221947 23927 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 8: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:04.215775 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:04.222553 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.132001 28534 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
W20260709 12:15:04.187537 28475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.005s	sys 0.024s
I20260709 12:15:04.224953 23751 leader_election.cc:400] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:04.228124 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.227244 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:04.202046 28503 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:04.201922 28488 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:04.215548 28494 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:04.222908 23778 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:04.234464 25176 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.238358 28501 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:04.212641 28532 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:04.169548 28370 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.260531 23779 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:04.240060 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.231498 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.263792 28539 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.264971 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.265614 25174 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:04.268033 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.269515 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.270694 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.233521 28417 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11.
I20260709 12:15:04.236953 28576 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:04.212828 28304 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.278818 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:04.252002 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.272127 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.227828 28402 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:04.234439 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.236390 28157 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:04.237239 28480 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:04.248843 28307 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:04.236610 28589 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:15:04.288408 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 54328.8us
I20260709 12:15:04.251538 28549 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:04.307116 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:04.252744 28407 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:04.304536 25175 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:04.309649 23825 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.273123 28494 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.254663 28551 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.254640 28492 consensus_queue.cc:1243] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:15:04.304306 28490 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.319388 28503 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:04.312773 28407 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.322741 28407 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:04.319902 28544 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.324535 28544 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.321938 28503 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.320029 28304 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:04.328161 28304 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.328534 28304 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:04.327677 28503 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.305343 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.275806 28164 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.333755 28164 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.315491 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.335973 28164 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:04.311932 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:04.303925 28576 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:04.336984 23778 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:04.337371 28576 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.338657 28407 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.341079 28496 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.252475 28588 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:15:04.352011 28584 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:04.352620 28584 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.353492 28584 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:04.288125 28478 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:04.356114 28589 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:04.357204 28589 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:04.359200 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.359674 28407 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.289590 28402 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.363094 28402 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:04.289024 28537 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:15:04.364459 28602 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:04.365688 28586 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:04.249111 28524 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.367384 28524 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.306602 28307 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:04.310458 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2809) took 1793 ms. Trace:
I20260709 12:15:04.371806 23784 rpcz_store.cc:276] 0709 12:15:02.517265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.774987 (+1257722us) service_pool.cc:224] Handling call
0709 12:15:03.899051 (+124064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:03.899776 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.310259 (+410483us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540041c150. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:04.310398 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250704640}
I20260709 12:15:04.337859 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:04.368513 28307 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.373646 23778 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:04.374456 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.376451 25174 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:04.377583 25172 raft_consensus.cc:2410] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:15:04.379101 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.286046 28501 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.298283 28431 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.286978 28157 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.387800 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:04.407510 23784 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:04.404210 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.411929 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2787) took 1271 ms. Trace:
I20260709 12:15:04.412442 23778 rpcz_store.cc:276] 0709 12:15:03.140439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.376413 (+1235974us) service_pool.cc:224] Handling call
0709 12:15:04.410925 (+ 34512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.411259 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.411309 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:04.411378 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:04.411438 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.411517 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.411810 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:04.416488 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.418859 28157 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.426093 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.427543 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.429030 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.415079 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2799) took 1124 ms. Trace:
I20260709 12:15:04.431069 23785 rpcz_store.cc:276] 0709 12:15:03.290725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.379460 (+1088735us) service_pool.cc:224] Handling call
0709 12:15:04.414519 (+ 35059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.414734 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.414759 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:04.414803 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:04.414841 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.414892 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.415008 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.433176 23785 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:15:04.436537 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.438987 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2833) took 1057 ms. Trace:
I20260709 12:15:04.439767 23785 rpcz_store.cc:276] 0709 12:15:03.381701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.432441 (+1050740us) service_pool.cc:224] Handling call
0709 12:15:04.432831 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.438440 (+  5609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.438846 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.448256 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.382382 28370 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:04.469347 23778 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:15:04.469851 23778 pending_rounds.cc:85] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 8
I20260709 12:15:04.470083 23778 pending_rounds.cc:107] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:15:04.470345 23778 raft_consensus.cc:2889] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:04.481587 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2832) took 1119 ms. Trace:
I20260709 12:15:04.483145 23778 rpcz_store.cc:276] 0709 12:15:03.362333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.413656 (+1051323us) service_pool.cc:224] Handling call
0709 12:15:04.468931 (+ 55275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.480911 (+ 11980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.481495 (+   584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.471763 28517 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:04.498729 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.499883 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:04.502417 28550 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:04.502890 28550 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.503796 28550 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.505285 28550 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.508194 28493 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:04.508725 28493 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.509649 28493 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.511214 28493 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:15:04.512197 28604 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:15:04.513959 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.514714 23858 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:04.515625 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.516402 23858 raft_consensus.cc:2410] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:04.517136 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.518362 23858 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:04.523440 23858 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:04.524559 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.528056 25176 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:04.547683 28045 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.553576 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2786) took 1413 ms. Trace:
I20260709 12:15:04.551201 28045 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:04.556061 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.554056 23781 rpcz_store.cc:276] 0709 12:15:03.140296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.092343 (+952047us) service_pool.cc:224] Handling call
0709 12:15:04.167119 (+ 74776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.167365 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.167391 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.167916 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.375437 (+207521us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:04.375540 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.375633 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:04.553337 (+177704us) raft_consensus.cc:1685] finished
0709 12:15:04.553358 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.553507 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":207231,"threads_started":1,"wal-append.queue_time_us":369412}
W20260709 12:15:04.554363 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.488111 23778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "102493ff11494b009f79963bb57be511"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:04.542866 28501 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.578523 23851 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:15:04.547324 28511 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.557765 28483 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.560657 28527 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:04.589583 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:04.592991 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 191289us
I20260709 12:15:04.554132 28517 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.606359 23859 raft_consensus.cc:2368] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:04.492414 28494 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.609756 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:04.611228 23778 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:04.616600 23778 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.618571 28615 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:15:04.621600 28603 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:15:04.624023 28517 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.626995 23778 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.630659 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:04.601512 28527 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.610898 28478 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:15:04.607879 28407 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:15:04.637092 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2782) took 1497 ms. Trace:
I20260709 12:15:04.638403 28511 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:04.640014 28507 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:04.606526 28402 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:04.619354 28599 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:04.644441 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2810) took 2127 ms. Trace:
I20260709 12:15:04.644829 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.669663 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.651549 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.653353 23749 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.607415 23857 raft_consensus.cc:2955] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 4 to 7, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:04.664575 23783 rpcz_store.cc:276] 0709 12:15:03.139548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.040009 (+900461us) service_pool.cc:224] Handling call
0709 12:15:04.040391 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.041018 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.041050 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.042045 (+   995us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.255782 (+213737us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:04.255831 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.255901 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:04.636848 (+380947us) raft_consensus.cc:1685] finished
0709 12:15:04.636875 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.637027 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"thread_start_us":212629,"threads_started":1,"wal-append.queue_time_us":443686}
I20260709 12:15:04.652346 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.673105 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.674443 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.668553 23898 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:04.679605 23749 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.602337 28483 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.694849 23898 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:04.698825 23783 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:04.699757 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 877) took 1041 ms. Trace:
I20260709 12:15:04.700465 23783 rpcz_store.cc:276] 0709 12:15:03.658595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.671713 (+1013118us) service_pool.cc:224] Handling call
0709 12:15:04.698615 (+ 26902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.699291 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.699545 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:04.669754 23776 rpcz_store.cc:276] 0709 12:15:02.517501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.821569 (+1304068us) service_pool.cc:224] Handling call
0709 12:15:03.821953 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:03.874068 (+ 52115us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:04.155816 (+281748us) log.cc:844] Serialized 14 byte log entry
0709 12:15:04.479799 (+323983us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.491679 (+ 11880us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.491863 (+   184us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:04.491898 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.491972 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:04.644223 (+152251us) raft_consensus.cc:1685] finished
0709 12:15:04.644244 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.644383 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51616,"prepare.queue_time_us":332259,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":8901,"raft.queue_time_us":54892,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":9596,"thread_start_us":498767,"threads_started":2,"wal-append.queue_time_us":417206}
I20260709 12:15:04.669191 28615 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.670100 28507 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.644412 28307 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:04.713150 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.676479 28487 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.694239 28480 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.694543 28045 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:15:04.702549 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 366199us
I20260709 12:15:04.710404 28511 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.675765 28402 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.748198 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:04.749047 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.693866 28304 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.746109 23783 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:04.738721 23856 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:04.697031 28485 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:04.749959 23931 raft_consensus.cc:2410] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:04.695542 28532 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:15:04.745641 28431 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:04.713781 28607 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:04.745460 25172 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:04.738564 28488 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:04.702627 28544 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:15:04.741724 28045 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.765903 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:04.765568 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2831) took 1416 ms. Trace:
I20260709 12:15:04.768520 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.769743 23898 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:04.774233 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.778496 23933 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:04.779373 23784 rpcz_store.cc:276] 0709 12:15:03.348914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.408242 (+1059328us) service_pool.cc:224] Handling call
0709 12:15:04.446462 (+ 38220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.446692 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:04.468282 (+ 21590us) log.cc:844] Serialized 14 byte log entry
0709 12:15:04.765240 (+296958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:04.765291 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:04.765337 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.765399 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.765504 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24766,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":95,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3581,"thread_start_us":296510,"threads_started":1}
I20260709 12:15:04.766403 27999 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.777370 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:04.783493 23751 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.782567 23856 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:04.762812 28621 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:04.777283 23855 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:04.787482 23926 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:04.792204 25177 raft_consensus.cc:606] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:04.768260 28417 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.784901 28407 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.789311 28498 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:04.774235 28511 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:04.836143 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:04.789489 23783 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:15:04.782564 28516 consensus_meta.cc:287] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 12:15:04.837523 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.839383 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.781610 28045 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:04.848466 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.772716 28517 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:15:04.785338 28607 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:04.786230 28544 consensus_queue.cc:1243] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:15:04.879545 28402 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:04.893980 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:04.896140 23783 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:15:04.896140 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.844996 28511 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:04.909727 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:04.910843 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 127437us
W20260709 12:15:04.911092 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 161167us
W20260709 12:15:04.911988 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:04.911391 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 75347.6us
I20260709 12:15:04.903301 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 872) took 1282 ms. Trace:
I20260709 12:15:04.906687 23926 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:04.882777 28407 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:04.834795 27999 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.773013 28507 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.921006 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.896185 28496 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:04.839870 28417 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.915752 23785 rpcz_store.cc:276] 0709 12:15:03.620482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.462558 (+842076us) service_pool.cc:224] Handling call
0709 12:15:04.462882 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.463072 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:04.752089 (+289017us) log.cc:844] Serialized 14 byte log entry
0709 12:15:04.902944 (+150855us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:04.902995 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:04.903041 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.903103 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.903239 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9035,"prepare.queue_time_us":26331,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2633,"thread_start_us":404642,"threads_started":2}
I20260709 12:15:04.837651 28621 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.845157 28498 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:04.899325 28607 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:04.934332 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:04.914772 28511 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:04.930575 28496 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:04.962345 23853 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:04.964272 28621 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:04.936563 28615 consensus_queue.cc:1059] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:04.966102 23782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 875) took 1344 ms. Trace:
I20260709 12:15:04.966511 23782 rpcz_store.cc:276] 0709 12:15:03.621352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.499620 (+878268us) service_pool.cc:224] Handling call
0709 12:15:04.499978 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.500300 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.500329 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.500812 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.768665 (+267853us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:04.768710 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.768771 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:04.965862 (+197091us) raft_consensus.cc:1685] finished
0709 12:15:04.965885 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:04.966028 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1468,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":267438,"threads_started":1,"wal-append.queue_time_us":460353}
I20260709 12:15:04.958899 28483 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:04.973928 28622 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:15:04.974179 28496 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.976889 28496 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:04.981323 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.982161 28498 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:04.984304 28619 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:15:04.986197 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:04.988334 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:04.962495 28607 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:04.992373 28511 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:04.994719 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 81918.1us
I20260709 12:15:05.001300 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 876) took 1379 ms. Trace:
I20260709 12:15:05.001768 23781 rpcz_store.cc:276] 0709 12:15:03.621607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.557301 (+935694us) service_pool.cc:224] Handling call
0709 12:15:04.573189 (+ 15888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.573593 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:04.590151 (+ 16558us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.000946 (+410795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:05.001002 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:05.001050 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.001117 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.001234 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17559,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":51824,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8612,"thread_start_us":410100,"threads_started":2,"wal-append.queue_time_us":389237}
W20260709 12:15:05.003068 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.011314 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:04.992259 28624 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:15:05.012070 23785 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:05.013396 23784 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:04.928635 28527 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.023548 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.029253 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:04.999737 28532 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.008878 28496 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.013075 28307 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:05.044373 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.047915 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:05.049979 28631 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:15:05.055104 28318 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:04.922009 28488 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.008376 28591 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:15:05.009915 28513 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:15:05.063685 23852 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46448 (request call id 664) took 1429 ms (client timeout 1733 ms). Trace:
W20260709 12:15:05.064201 23852 rpcz_store.cc:269] 0709 12:15:03.634162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.658106 (+ 23944us) service_pool.cc:224] Handling call
0709 12:15:05.063430 (+1405324us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540037ee54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:05.063610 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306779008}
I20260709 12:15:05.065671 23854 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:05.068609 28420 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:05.069057 28420 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:05.070010 28420 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.071099 28420 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:05.082779 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 155348us
I20260709 12:15:05.039048 28304 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.097502 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:05.085947 28639 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.037117 28532 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.042028 28496 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.087762 28511 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.122622 28549 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:05.125245 28612 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.624s	user 0.002s	sys 0.001s
I20260709 12:15:05.147992 25175 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 1107) took 1060 ms. Trace:
I20260709 12:15:05.148449 25175 rpcz_store.cc:276] 0709 12:15:04.087726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.087861 (+   135us) service_pool.cc:224] Handling call
0709 12:15:05.147750 (+1059889us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540064cdd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:05.147897 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495821568}
I20260709 12:15:05.150238 28487 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.151027 23858 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:05.151214 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 670) took 1056 ms. Trace:
I20260709 12:15:05.151578 23854 rpcz_store.cc:276] 0709 12:15:04.095014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.241482 (+146468us) service_pool.cc:224] Handling call
0709 12:15:05.065280 (+823798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:05.066134 (+   854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.150992 (+ 84858us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b540037ee50. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:05.151140 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893790464}
I20260709 12:15:05.155990 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:05.157263 28597 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:05.115312 28493 consensus_meta.cc:287] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.004s
I20260709 12:15:05.163578 23781 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:15:05.158998 28524 consensus_meta.cc:287] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.791s	user 0.013s	sys 0.000s
I20260709 12:15:05.168663 28591 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.115168 28639 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:05.193145 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.158694 28589 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:05.173272 28593 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.853s	user 0.006s	sys 0.000s
I20260709 12:15:05.197841 28513 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.198482 28513 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.163192 28480 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.200636 28642 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.193234 28650 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.201798 23782 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:05.201958 28650 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.201278 28642 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.204290 28650 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.204870 28650 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.205485 28496 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.205365 28650 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.205751 28642 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.203378 28513 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.204308 23933 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:05.208302 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:05.208578 28550 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.000s
I20260709 12:15:05.180306 28638 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.209484 28638 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.210485 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.211181 23932 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:05.195763 28591 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.159910 28597 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:05.216558 28591 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.216897 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.217896 23932 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:15:05.218511 28600 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.899s	user 0.003s	sys 0.000s
I20260709 12:15:05.226787 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 867) took 2442 ms. Trace:
I20260709 12:15:05.218811 28532 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.219245 28417 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.219657 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.227499 28661 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:05.221685 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.229269 28661 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.230101 23933 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:05.196238 28589 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:05.196148 27999 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:05.231706 28417 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:05.231889 28597 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.203459 28639 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.233660 28638 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.234239 23778 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.234414 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 669) took 1139 ms. Trace:
I20260709 12:15:05.271055 23857 rpcz_store.cc:276] 0709 12:15:04.094765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.212687 (+117922us) service_pool.cc:224] Handling call
0709 12:15:04.213283 (+   596us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:04.218147 (+  4864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.218181 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:04.240713 (+ 22532us) log.cc:844] Serialized 520 byte log entry
0709 12:15:04.579631 (+338918us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:05.014244 (+434613us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.029432 (+ 15188us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.233768 (+204336us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b180034bd60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:15:05.233986 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.234060 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:05.234207 (+   147us) raft_consensus.cc:1685] finished
0709 12:15:05.234223 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.234357 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":209091,"prepare.queue_time_us":16721,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":207404,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":20146,"spinlock_wait_cycles":3578752,"thread_start_us":277217,"threads_started":1,"wal-append.queue_time_us":414882}
I20260709 12:15:05.194212 28045 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.235461 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.227200 23780 rpcz_store.cc:276] 0709 12:15:02.784004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.960606 (+1176602us) service_pool.cc:224] Handling call
0709 12:15:03.988109 (+ 27503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:03.988510 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:03.988543 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.004987 (+ 16444us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.216029 (+211042us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:04.216078 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.216150 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:05.217953 (+1001803us) spinlock_profiling.cc:243] Waited 58.2 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:05.226453 (+  8500us) raft_consensus.cc:1685] finished
0709 12:15:05.226478 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.226720 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8110,"spinlock_wait_cycles":133820672,"thread_start_us":201424,"threads_started":1,"wal-append.queue_time_us":299127}
W20260709 12:15:05.236406 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 237098us
W20260709 12:15:05.236764 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.235879 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 152049us
I20260709 12:15:05.237442 28650 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.237807 28157 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:05.296873 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:05.237964 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 220855us
I20260709 12:15:05.240477 28586 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.242511 28507 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:05.244812 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2783) took 2105 ms. Trace:
I20260709 12:15:05.348453 23777 rpcz_store.cc:276] 0709 12:15:03.139814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.043747 (+903933us) service_pool.cc:224] Handling call
0709 12:15:04.044040 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.044280 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.044306 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:04.116819 (+ 72513us) log.cc:844] Serialized 26 byte log entry
0709 12:15:04.382021 (+265202us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:04.382061 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:04.382130 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:05.124646 (+742516us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:05.244455 (+119809us) raft_consensus.cc:1685] finished
0709 12:15:05.244482 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.244744 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53366016,"thread_start_us":169412,"threads_started":1,"wal-append.queue_time_us":320690}
I20260709 12:15:05.247812 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2808) took 2730 ms. Trace:
I20260709 12:15:05.349917 23779 rpcz_store.cc:276] 0709 12:15:02.517011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.772824 (+1255813us) service_pool.cc:224] Handling call
0709 12:15:03.790347 (+ 17523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.261118 (+470771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.247483 (+986365us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400109854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:05.247721 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080602880}
I20260709 12:15:05.253314 23859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1437) took 1165 ms. Trace:
I20260709 12:15:05.350965 23859 rpcz_store.cc:276] 0709 12:15:04.088159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.088283 (+   124us) service_pool.cc:224] Handling call
0709 12:15:05.253097 (+1164814us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400386b54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:05.253241 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000935808}
I20260709 12:15:05.256316 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:05.352345 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 96101.4us
I20260709 12:15:05.354025 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:05.259053 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.021s	sys 0.014s
W20260709 12:15:05.355806 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.178s	user 0.022s	sys 0.015s
W20260709 12:15:05.356456 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 1.18477e+06us
I20260709 12:15:05.357908 28620 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:05.358934 28494 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.283207 23857 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:15:05.284034 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.285102 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:05.361243 23858 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:05.285869 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.286701 23932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98e5bc86804f4f979f58779caa696814"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:05.362628 23932 raft_consensus.cc:606] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:05.363015 23932 raft_consensus.cc:614] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:05.280093 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.273962 28589 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.364210 23784 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.364391 28589 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:05.367058 23932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:05.269742 28417 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.367750 23932 raft_consensus.cc:606] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:05.289003 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.289848 28597 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:05.290215 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40134 (request call id 873) took 1652 ms. Trace:
I20260709 12:15:05.291952 28045 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.293128 25114 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:05.369228 28417 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:05.369257 23778 rpcz_store.cc:276] 0709 12:15:03.620787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.484213 (+863426us) service_pool.cc:224] Handling call
0709 12:15:05.223425 (+739212us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54001ef854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7a9a3d 00007fb2b2779cd0 00007fb2abcfe254 00007fb2abcfe1e2 00007fb2abcfe111 00007fb2abcfe08c 00007fb2abcfd342 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:05.273447 (+ 50022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104886912}
I20260709 12:15:05.371331 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:05.372481 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.349311 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2838) took 1219 ms. Trace:
I20260709 12:15:05.376863 23783 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:15:05.377137 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.377251 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:05.355386 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:05.379444 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:05.384209 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.384922 23854 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:15:05.386859 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.388346 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.383422 23784 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:05.389519 23751 leader_election.cc:400] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:05.389850 23751 leader_election.cc:403] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:05.404135 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:05.402720 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.410553 23925 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:05.412688 23934 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:15:05.412981 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:05.414819 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:05.385756 28620 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:15:05.411820 23931 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:05.416726 23929 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:05.411850 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.387933 28639 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.420049 28639 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.420826 28516 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:05.421252 28516 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:05.421612 28516 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:05.421677 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:05.422132 28639 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.422300 23929 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.412722 23856 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:15:05.425799 23932 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:05.425799 23781 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:05.425437 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.384411 28402 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:05.386225 28659 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.298111 28475 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 12:15:05.385102 28638 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.407217 28532 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:05.461953 28289 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:05.462441 28289 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:05.375486 23776 rpcz_store.cc:276] 0709 12:15:04.129681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.709699 (+580018us) service_pool.cc:224] Handling call
0709 12:15:04.710055 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.710261 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:15:04.739149 (+ 28888us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.157662 (+418513us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b1800388480. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:15:05.348956 (+191294us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:05.349015 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:05.349060 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.349130 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.349254 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29512,"prepare.queue_time_us":30845,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1957,"raft.queue_time_us":30008,"spinlock_wait_cycles":5187968,"thread_start_us":161461,"threads_started":1}
I20260709 12:15:05.445746 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.390625 28494 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:05.440862 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.473340 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.473897 23929 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:05.474040 23855 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:05.438715 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:05.444630 28511 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:05.445180 28621 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:05.427309 28609 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.446357 28618 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:05.388831 28335 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:15:05.488343 23855 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:05.487920 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 49288.6us
I20260709 12:15:05.499300 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:05.436125 28524 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:15:05.436311 28591 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:05.375382 28586 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.458426 28490 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:05.433332 28431 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:15:05.459986 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:05.468243 23932 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:05.471972 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 79809.2us
I20260709 12:15:05.463398 28517 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:05.436290 28493 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:15:05.483352 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:05.482209 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.563172 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:05.484302 23929 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:15:05.564070 23929 pending_rounds.cc:85] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 9
I20260709 12:15:05.564445 23929 pending_rounds.cc:107] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:15:05.564738 23929 raft_consensus.cc:2889] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:05.485355 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:05.454767 28492 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:05.503713 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 210632us
W20260709 12:15:05.568414 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 83145.1us
I20260709 12:15:05.569311 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:05.504946 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:05.505586 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:05.570578 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:05.572237 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 88974.9us
I20260709 12:15:05.573902 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.570921 23852 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:05.575337 23784 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:05.576249 23853 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:05.488173 28618 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:05.511683 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:05.480794 28475 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.510444 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:05.644912 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:05.577828 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:05.471920 28638 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.580090 23785 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:05.585086 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:05.593816 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:05.649289 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 71591.2us
I20260709 12:15:05.662914 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:05.663862 25172 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.466467 28659 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.662123 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:05.649335 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:05.655851 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71624.1us
I20260709 12:15:05.648242 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.649801 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:05.501647 28045 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:05.504570 28591 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.672116 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:05.679044 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:05.683177 23784 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.678020 23779 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
W20260709 12:15:05.680261 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 86557us
I20260709 12:15:05.516006 28517 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:05.774529 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59015.4us
I20260709 12:15:05.686465 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:05.776916 23784 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:05.686530 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:05.758951 23852 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:05.583643 28478 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:15:05.782532 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 96141us
I20260709 12:15:05.567359 28492 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:05.645020 28511 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:05.659768 28620 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:05.502228 28609 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.584479 28646 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:05.561779 28576 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.124s
I20260709 12:15:05.673717 28532 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:05.772868 28591 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.561295 28550 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.213s
I20260709 12:15:05.675879 28664 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.584656 28607 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.199s
I20260709 12:15:05.803308 25172 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
W20260709 12:15:05.728020 28640 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:15:05.576102 28597 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.212s
I20260709 12:15:05.732551 28551 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:15:05.786999 28164 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.782352 28488 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:05.687316 28420 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.687465 28370 consensus_queue.cc:1059] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:15:05.789953 28407 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:05.789595 28513 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.695487 28475 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.819689 28496 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.819235 28639 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.818125 28661 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:05.789196 28503 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:15:05.818332 28478 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:05.850574 28576 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:05.849502 28650 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.876426 28665 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.882112 28494 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:15:05.885138 28407 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:05.887722 28488 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:05.883826 28668 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.897302 28639 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.883072 28307 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:15:05.851558 28664 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.889827 28513 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:05.882988 28164 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:05.938537 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 224274us
I20260709 12:15:05.876300 28517 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.908452 28576 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:05.884809 28420 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.928498 28407 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:05.930346 28650 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.895500 28496 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:05.950891 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.971582 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 296226us
I20260709 12:15:05.976609 25172 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:05.890515 28618 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.978857 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:05.947019 28576 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:05.982072 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:05.962427 28586 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:05.925158 28665 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.986840 28550 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:05.989007 28496 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:05.992229 28507 consensus_meta.cc:287] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.000s
I20260709 12:15:05.977269 28493 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:05.996104 28666 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:15:05.995509 28591 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:05.998004 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.043888 25172 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:06.001674 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.042944 23859 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:06.011989 28164 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.012372 28609 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:05.920542 28492 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.014163 28513 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.014720 28488 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.016113 28478 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.017596 28659 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:06.018934 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:06.025897 28157 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.001s
I20260709 12:15:05.941684 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:05.973615 28517 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.057690 23860 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:15:06.040163 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 264192us
I20260709 12:15:06.005663 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:06.041203 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.061622 25177 raft_consensus.cc:3055] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:06.062026 25177 raft_consensus.cc:740] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:06.062889 25177 consensus_queue.cc:260] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.063922 25177 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:06.051925 23781 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:06.038856 28550 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.076244 25175 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:06.076956 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.085072 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.089263 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.090418 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.092137 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.094022 23783 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:15:06.098743 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.103495 25177 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:06.042689 28551 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:15:06.074501 28550 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.119557 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.049587 28493 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.125831 23781 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:06.126363 23931 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:06.127019 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 616589us
I20260709 12:15:06.103739 28589 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:06.136718 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:06.137259 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.137499 23934 raft_consensus.cc:2393] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:06.138046 23857 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:06.100850 28641 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:06.136883 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.101263 28496 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.132790 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.153908 23931 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:06.154314 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.154824 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.139665 23852 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: [12.13-12.13]
I20260709 12:15:06.140491 23855 raft_consensus.cc:1240] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:06.155437 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.109297 28659 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:06.134159 28045 raft_consensus.cc:993] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:06.130080 28517 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:06.139089 28516 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.717s	user 0.013s	sys 0.000s
I20260709 12:15:06.160742 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.173779 23778 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:06.160900 23931 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:15:06.167415 25177 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:06.142098 23933 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:06.108381 28157 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:06.167431 23749 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:06.169499 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.165145 28683 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.181226 25113 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.133816 28475 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.183849 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.139424 28669 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.185017 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.182997 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.142835 28589 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:06.188217 23781 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:06.189218 23931 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:06.190606 23931 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:15:06.190968 23931 pending_rounds.cc:85] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 9
I20260709 12:15:06.191192 23931 pending_rounds.cc:107] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:15:06.191426 23931 raft_consensus.cc:2889] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:06.159370 28659 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.170614 28681 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.165998 28490 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:06.204298 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.152139 28668 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:15:06.214846 23933 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:06.183845 28513 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:06.152860 28507 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:15:06.187913 28586 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.204974 28045 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.212813 28475 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.212911 28659 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.215037 28641 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.214766 28639 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.183420 28516 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.260589 28513 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:06.254503 28664 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.189853 28669 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.262526 28688 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:06.204301 28686 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:15:06.260651 28694 raft_consensus.cc:1081] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:06.218637 28420 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.276504 28475 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.262645 28490 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:06.293358 23928 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:06.288146 28674 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:06.279186 28685 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.286368 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.264499 28045 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.261466 28586 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.224556 28681 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.298470 28688 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.298839 28591 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.281999 28641 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.278070 28589 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.310477 28665 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.303238 28157 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:15:06.318681 28688 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:06.331400 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 161467us
W20260709 12:15:06.328980 28680 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:15:06.338548 23781 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:06.336547 28614 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:06.326344 28493 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.343422 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.328680 28370 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
W20260709 12:15:06.339075 28682 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:15:06.353955 23855 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:06.361440 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.312408 28685 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.365356 23855 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:06.366003 23855 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:06.366410 23855 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.367202 28045 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.370639 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.371330 23783 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:06.372447 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.367193 28669 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.374310 28586 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.376528 28639 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.377074 28639 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.377899 28496 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.378466 28496 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.379278 28639 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.340574 28614 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:06.427870 28661 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:15:06.427496 28614 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.428668 28614 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:06.429708 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.380517 28496 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.380795 25175 raft_consensus.cc:1275] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:06.436673 23855 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.437666 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2861) took 1017 ms. Trace:
I20260709 12:15:06.438078 23928 rpcz_store.cc:276] 0709 12:15:05.419813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.420754 (+   941us) service_pool.cc:224] Handling call
0709 12:15:05.421091 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:06.303999 (+882908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.437406 (+133407us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b5400203fd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:06.437601 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005496192}
I20260709 12:15:06.381045 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.440425 23855 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.382097 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:06.460727 28289 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.461959 28289 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.463763 28576 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.340476 28490 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.466102 28576 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.380949 28164 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.467926 28164 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.470404 28164 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.385708 28665 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:06.376305 28549 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:06.380549 28475 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.389138 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:06.477267 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 292472us
I20260709 12:15:06.389046 28645 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.396023 28652 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.398911 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:06.399796 28516 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.400941 28681 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.403357 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:06.490717 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 87424.1us
I20260709 12:15:06.404500 28636 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:06.404896 28487 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:06.405223 28678 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:06.340731 28407 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.406001 28638 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.334685 28513 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.406703 28702 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:06.353102 28641 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.407471 28520 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:06.409221 28402 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:06.409600 28685 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:06.411240 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.416121 28694 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.417770 28662 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.841s	user 0.003s	sys 0.001s
I20260709 12:15:06.420362 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:06.423537 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.426349 28517 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:15:06.440375 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59416.3us
I20260709 12:15:06.550477 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.381582 28669 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:06.441543 28493 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:06.443755 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.445137 28683 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:06.551445 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.447705 28550 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.399267 28659 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.562330 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.564081 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.451079 28639 consensus_queue.cc:1048] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:06.473265 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.473968 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.462917 28688 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.474843 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.475145 23856 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:06.480569 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.573135 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:06.481552 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82710.9us
I20260709 12:15:06.573951 23926 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:06.503741 28157 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:06.542676 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.580483 23855 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:06.581444 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.572396 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.582600 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.583201 23855 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:06.584071 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.584447 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:06.584914 23855 raft_consensus.cc:2410] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:15:06.501423 28621 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.545969 28402 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:06.547964 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.018s
W20260709 12:15:06.595180 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.006s	sys 0.018s
W20260709 12:15:06.549494 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 364515us
I20260709 12:15:06.534550 28702 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.478222 28549 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.555326 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:06.555635 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.602383 23931 raft_consensus.cc:3055] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:06.563419 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.604099 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.604617 23930 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:06.565616 23776 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:06.490890 28645 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.567513 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.568485 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.557588 28597 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:06.477716 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:06.575111 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.569938 28532 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.613489 23927 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:06.500027 28652 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.540570 28475 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:15:06.572430 28496 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.602738 23931 raft_consensus.cc:740] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:06.546738 28641 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.566818 28164 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.609998 23898 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:06.546214 28520 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:06.612031 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44622.9us
I20260709 12:15:06.609479 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.598340 28607 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.596536 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.615207 25171 raft_consensus.cc:2393] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:06.572144 28676 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.578207 28681 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:06.581121 28517 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.580288 28157 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.665161 28157 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:06.666592 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 70169.2us
I20260709 12:15:06.598948 28620 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.628949 28638 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.614116 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.642457 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.602170 28702 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.636606 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.607331 28045 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:06.550336 28694 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.614821 28692 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:06.627857 28532 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.633922 28493 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.642479 28641 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.653316 28520 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.655320 28652 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.658016 28407 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.642845 28691 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:06.648128 28696 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:06.697346 28657 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.707644 28696 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:06.661029 28402 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.712881 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:06.713371 28402 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.714221 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.643539 28164 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.662256 28681 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.715997 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.717960 28164 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.643422 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.718817 23777 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:15:06.671157 28534 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:06.720610 28534 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.619504 28478 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:06.618734 23931 consensus_queue.cc:260] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:06.675045 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 88200.9us
I20260709 12:15:06.638490 28496 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.679272 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:06.730563 25177 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.730849 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.731500 25169 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 5.
I20260709 12:15:06.607332 28646 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 12:15:06.700289 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 63770us
I20260709 12:15:06.737274 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.701889 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.704141 23857 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:06.622042 28597 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:06.706140 28706 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:15:06.708006 28657 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.711180 23925 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:06.716208 28681 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.725483 23931 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.671509 23783 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:06.691790 28665 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:15:06.691493 28576 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:15:06.727160 23825 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:06.677052 23784 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:06.728565 28549 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.729290 28694 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.735875 23928 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:06.739477 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 93095.5us
I20260709 12:15:06.721511 28478 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.669433 28607 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.733886 28641 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.726785 28490 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:06.723449 28532 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.682955 28638 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:06.770968 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.752238 28691 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.753769 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 117712us
I20260709 12:15:06.703480 28493 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.773582 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:06.772238 25177 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:06.729211 28683 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:15:06.795642 23824 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:06.768033 28537 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:06.769603 28478 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.794061 23751 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:06.797221 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.801731 23781 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:06.750254 28696 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.793787 28691 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.826560 23931 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:06.816087 28641 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:06.857764 28702 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.860321 28650 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.861738 23785 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:15:06.863060 28492 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.864959 28493 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:06.865793 28621 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:06.866235 28621 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:06.868211 28691 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.870288 28702 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.799557 28550 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:06.850098 28684 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.576s	user 0.004s	sys 0.000s
I20260709 12:15:06.872782 28638 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.873386 28645 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.874186 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3313) took 2379 ms. Trace:
I20260709 12:15:06.874917 28496 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.876575 28597 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:06.876945 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.879575 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.880035 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.880668 25176 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:06.879441 25170 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:06.881299 28657 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:06.881791 25168 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:06.882050 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.882187 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:06.882539 28714 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:15:06.883505 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.867167 28621 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.884311 28657 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:06.885819 28618 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:06.885726 28657 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:06.886425 28657 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:06.815207 28402 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:06.887054 28691 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:06.887287 28657 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.887423 28402 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.887996 28691 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:06.888417 28691 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:06.888543 28657 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.889070 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.889525 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:06.890324 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.892306 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.893718 28607 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.894914 28607 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:06.895530 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.884641 28621 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.895936 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.896636 25168 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:06.898100 28431 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:06.899420 28524 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:06.899883 28524 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:06.899724 28641 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:06.900343 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:06.874565 23860 rpcz_store.cc:276] 0709 12:15:04.494691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.589132 (+ 94441us) service_pool.cc:224] Handling call
0709 12:15:04.605891 (+ 16759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:06.058231 (+1452340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.873911 (+815680us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54003c0d54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:06.874120 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3337786112}
I20260709 12:15:06.901340 28550 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:06.901175 28641 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.901759 28550 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.902141 28550 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.902823 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.903504 23776 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:06.884796 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.904444 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:06.905282 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.905483 23824 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.783928 28716 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:06.842983 23931 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:06.844125 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.829617 28692 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.848272 28715 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:15:06.849166 28710 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:15:06.849891 28707 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.818203 28659 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:06.851200 28695 log.cc:927] Time spent T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.525s	user 0.000s	sys 0.002s
I20260709 12:15:06.852576 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:06.853768 28713 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:15:06.906417 25177 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:06.924077 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.923789 23858 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:06.906836 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.925508 23930 raft_consensus.cc:2393] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:06.925359 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:06.960393 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:06.961542 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.961671 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.962410 25171 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:06.957410 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:06.926674 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 132672us
I20260709 12:15:06.963798 23778 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:06.964391 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:06.796963 28164 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:06.926919 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.889390 28691 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.972756 28157 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.973856 23784 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:06.924177 28716 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:06.925877 28045 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.942673 28487 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:06.945890 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.790946 28615 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:15:06.947129 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.949466 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.946842 28692 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:06.971851 28164 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.942263 28417 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:06.899976 28646 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.955001 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.962618 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.967829 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.967937 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.968142 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.967366 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:06.968778 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:06.977384 23853 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:06.945111 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:06.947933 28678 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:06.910022 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:06.979012 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.972306 28407 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.947324 28683 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.968858 28609 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:06.951611 28532 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:07.040517 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.980468 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:06.984557 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:06.990195 28402 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:07.002511 23927 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:07.001953 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.003489 23931 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:06.980896 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.075162 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.003998 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:07.004427 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.005174 23858 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:07.005615 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.078867 23851 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:07.006510 23857 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:07.008077 25169 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:07.081008 23778 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:07.082362 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:07.082919 23778 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.008370 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:07.084312 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.073606 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.085600 23855 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:07.010360 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.019964 23776 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:06.909446 28607 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:15:07.008780 23932 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:07.026345 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1456) took 1586 ms. Trace:
I20260709 12:15:06.982239 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.069860 23930 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:07.010828 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.090670 23856 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:07.021369 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.001376 25177 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:07.088037 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.093174 23776 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:07.089172 23854 rpcz_store.cc:276] 0709 12:15:05.439527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.439702 (+   175us) service_pool.cc:224] Handling call
0709 12:15:05.440048 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:05.478371 (+ 38323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:05.478404 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:05.504594 (+ 26190us) log.cc:844] Serialized 26 byte log entry
0709 12:15:06.372812 (+868218us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b18003b2120. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:06.611239 (+238427us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:06.611271 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.611321 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:07.025963 (+414642us) raft_consensus.cc:1685] finished
0709 12:15:07.025987 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.026271 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28631,"raft.queue_time_us":38715,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":21910,"spinlock_wait_cycles":3219840,"thread_start_us":238262,"threads_started":1,"wal-append.queue_time_us":1185805}
I20260709 12:15:07.092785 23898 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:06.973963 28691 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.064800 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:07.113811 23929 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:06.985348 28164 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.040067 28487 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.998399 28589 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.987046 28417 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.125041 28417 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.021245 28407 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.951488 28707 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.128948 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:07.129623 23930 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:06.979161 28716 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:07.132054 28688 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.062999 28685 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.095023 28494 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.132920 28685 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.133327 28494 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.006665 28657 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:07.136364 28490 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:07.136042 28657 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.136968 28490 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.137501 28490 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.039969 28045 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.008260 28678 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.139887 28678 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.071921 28615 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.142200 28615 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:06.975014 28157 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.068953 28532 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.115739 28675 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:15:07.013491 28705 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.469s	user 0.002s	sys 0.000s
I20260709 12:15:07.000914 28646 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.116241 28534 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:07.160943 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.021733 28609 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.007296 28692 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:06.981297 28493 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.139351 28657 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.188930 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.180226 23778 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:06.980309 28370 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.172s
I20260709 12:15:07.193205 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.195268 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.152885 28487 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.197638 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.200992 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.093472 28550 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.204710 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.204461 23928 raft_consensus.cc:2410] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:07.190083 23857 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:07.206672 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.109266 25169 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:07.207417 23928 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:07.174675 28488 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:07.204380 23776 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:07.009765 28681 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.208505 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.282158 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.283640 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:07.020135 28683 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.181368 28532 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.231413 23852 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:07.288614 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.289314 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.298108 23927 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:07.191170 28675 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:07.298892 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.151401 28478 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.199015 28615 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.202795 28692 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:07.110527 28524 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.197233 28534 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.230858 28492 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:15:07.293486 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:07.331342 23779 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:15:07.338423 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 45013.2us
I20260709 12:15:07.332490 25171 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:07.334355 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2873) took 1024 ms. Trace:
I20260709 12:15:07.288970 28496 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.295400 28537 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:07.210911 28704 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:07.279645 28711 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.310575 28685 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:07.284781 28659 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.326359 23778 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:07.315856 23785 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:07.335466 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.198789 28609 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:07.287616 28488 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:07.312075 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.390201 23776 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:07.331915 25114 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:07.385319 28711 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.378918 28704 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.340025 23934 rpcz_store.cc:276] 0709 12:15:06.309808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.309977 (+   169us) service_pool.cc:224] Handling call
0709 12:15:06.322002 (+ 12025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:06.322258 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:06.322285 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:06.327267 (+  4982us) log.cc:844] Serialized 26 byte log entry
0709 12:15:06.364829 (+ 37562us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:06.364871 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.364945 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:07.012767 (+647822us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:07.307703 (+294936us) raft_consensus.cc:1685] finished
0709 12:15:07.307729 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.334284 (+ 26555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6410624,"thread_start_us":37192,"threads_started":1,"wal-append.queue_time_us":176374}
I20260709 12:15:07.285908 28657 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:07.394055 28657 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.394624 28657 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.395000 23934 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:15:07.395453 23934 pending_rounds.cc:85] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 7
I20260709 12:15:07.395731 23934 pending_rounds.cc:107] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:15:07.395956 23934 raft_consensus.cc:2889] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:07.385591 28417 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.393671 28704 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.402849 28417 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:07.404556 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72725.1us
I20260709 12:15:07.405710 28318 raft_consensus.cc:2804] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:07.406451 28318 raft_consensus.cc:697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:07.406631 28370 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:07.407902 28318 consensus_queue.cc:237] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.410000 23930 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:07.308549 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:07.412999 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:07.413650 23928 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.415149 23776 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:07.203114 28493 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:07.352682 28513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.031s	sys 0.048s
W20260709 12:15:07.417315 28513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.031s	sys 0.048s
I20260709 12:15:07.331727 28494 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.393364 28534 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.420065 28478 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.378876 28537 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.421048 28537 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.423178 28493 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:07.425220 28550 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:07.430440 28657 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:07.340727 28719 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:15:07.432628 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.350777 28645 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:07.441843 28402 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:07.339398 28669 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:15:07.456940 25177 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:07.426724 28045 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:07.381877 28496 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.434621 28659 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:07.480432 28621 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.006s
I20260709 12:15:07.505523 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.516767 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:07.541615 25172 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.026s	sys 0.079s
W20260709 12:15:07.542582 25172 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.026s	sys 0.079s
I20260709 12:15:07.480324 28645 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.543637 28645 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.546991 23785 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:07.500962 28494 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:07.549153 28490 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.549958 28494 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.551909 28638 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.552739 28638 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.554390 28704 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.458634 28675 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:07.554410 28719 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:07.552778 28681 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:07.559716 28704 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.552819 28494 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.556036 28488 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.563009 28488 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.562541 28164 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.570971 28490 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.575848 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:07.577534 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.547205 28407 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:07.584794 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 520061us
I20260709 12:15:07.592979 23857 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:07.593984 23782 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:07.572511 28544 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:07.594519 23783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.005s	sys 0.009s
W20260709 12:15:07.596918 23783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.005s	sys 0.009s
I20260709 12:15:07.584793 28665 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:07.614262 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.615089 23857 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:07.594985 28646 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:07.619059 23823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.004s	sys 0.014s
W20260709 12:15:07.623016 23823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.005s	sys 0.014s
I20260709 12:15:07.628469 23928 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:07.612403 28407 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.629829 28707 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:07.596299 28507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.002s	sys 0.012s
W20260709 12:15:07.651479 23784 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.028s	sys 0.073s
W20260709 12:15:07.653929 23784 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.028s	sys 0.073s
W20260709 12:15:07.654448 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.655443 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.641556 28646 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.672648 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.672266 23854 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:07.675881 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.676563 23854 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:07.677733 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.678049 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.678546 23854 raft_consensus.cc:2410] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:15:07.678726 23855 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:07.651177 28507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.003s	sys 0.012s
I20260709 12:15:07.682497 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.647888 28609 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.696722 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:07.697212 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:07.698750 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:07.700377 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:07.702824 25172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 666) took 1023 ms. Trace:
I20260709 12:15:07.698010 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:07.704079 28657 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:07.727193 25169 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:07.715826 25119 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.725615 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:07.702055 28728 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:15:07.651268 28407 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:07.712395 28589 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.588s	user 0.017s	sys 0.004s
W20260709 12:15:07.707124 28618 consensus_meta.cc:287] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.821s	user 0.005s	sys 0.008s
I20260709 12:15:07.712387 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.737970 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:07.739681 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.744258 25175 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:07.743523 25119 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:07.732921 28730 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:15:07.682025 28646 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.736488 28045 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:07.735507 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.749352 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.723197 28704 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:07.723438 28741 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.749203 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:07.724491 28707 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:15:07.771154 28589 raft_consensus.cc:2764] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:07.790961 28741 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:07.746163 28318 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:07.728919 28681 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:15:07.746292 28620 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.782544 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:07.784857 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:07.803800 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 371246us
W20260709 12:15:07.786378 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 50963.7us
I20260709 12:15:07.749294 28711 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:07.727738 25172 rpcz_store.cc:276] 0709 12:15:06.679175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.679784 (+   609us) service_pool.cc:224] Handling call
0709 12:15:06.680105 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:06.700166 (+ 20061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:06.700211 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:06.710706 (+ 10495us) log.cc:844] Serialized 26 byte log entry
0709 12:15:06.903899 (+193193us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b1800454d20. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:07.542928 (+639029us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:07.542973 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.543039 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:07.686433 (+143394us) raft_consensus.cc:1685] finished
0709 12:15:07.686455 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.702728 (+ 16273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19019,"raft.queue_time_us":17150,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3079,"spinlock_wait_cycles":11256960,"thread_start_us":638852,"threads_started":1,"wal-append.queue_time_us":946442}
I20260709 12:15:07.746377 28650 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:07.771775 28494 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:07.791849 28589 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:07.770027 23855 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:07.793754 28704 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:07.796334 28335 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:07.742118 28685 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.804879 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:07.805102 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.805501 23856 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:07.775911 28549 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:07.812286 23825 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:07.749980 28576 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:15:07.869131 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 50572.7us
W20260709 12:15:07.839232 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.820130 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.869616 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.871938 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1357) took 1037 ms. Trace:
I20260709 12:15:07.872263 23929 rpcz_store.cc:276] 0709 12:15:06.834581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.323172 (+488591us) service_pool.cc:224] Handling call
0709 12:15:07.323483 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.323710 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:07.612767 (+289057us) log.cc:844] Serialized 14 byte log entry
0709 12:15:07.869866 (+257099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:07.869922 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:07.869966 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.870027 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.871878 (+  1851us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2434,"prepare.queue_time_us":182761,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"thread_start_us":542918,"threads_started":2}
I20260709 12:15:07.873644 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:07.874487 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 54436.8us
W20260709 12:15:07.878674 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 73628.5us
I20260709 12:15:07.879663 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.885540 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:07.898137 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 85923.4us
I20260709 12:15:07.902997 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 897) took 1191 ms. Trace:
I20260709 12:15:07.909044 25172 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:07.921454 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:07.903530 23777 rpcz_store.cc:276] 0709 12:15:06.712032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.719476 (+  7444us) service_pool.cc:224] Handling call
0709 12:15:06.719872 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:06.734737 (+ 14865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:06.734769 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:06.757280 (+ 22511us) log.cc:844] Serialized 52 byte log entry
0709 12:15:07.078634 (+321354us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:07.123294 (+ 44660us) log.cc:844] Serialized 14 byte log entry
0709 12:15:07.123541 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.131909 (+  8368us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:07.902716 (+770807us) raft_consensus.cc:1685] finished
0709 12:15:07.902743 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.902915 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14444,"prepare.queue_time_us":48910,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":16716,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":14992,"thread_start_us":284865,"threads_started":1,"wal-append.queue_time_us":815265}
I20260709 12:15:07.923381 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:07.870805 28487 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.003s
I20260709 12:15:07.926806 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:07.925870 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:07.818333 28494 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.933549 23749 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:07.818578 28589 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:07.866722 28688 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.936825 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:07.821216 28686 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:07.869350 28045 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:07.907120 28681 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:08.002251 23929 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:08.003803 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:08.004451 23929 raft_consensus.cc:3060] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.906631 28711 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:07.962105 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2839) took 1053 ms. Trace:
I20260709 12:15:08.013649 28609 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:08.013893 28736 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:15:07.869472 28618 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:15:08.014091 23784 rpcz_store.cc:276] 0709 12:15:06.908494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.974650 (+ 66156us) service_pool.cc:224] Handling call
0709 12:15:06.977176 (+  2526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:06.977402 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:07.671814 (+694412us) log.cc:844] Serialized 14 byte log entry
0709 12:15:07.961646 (+289832us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:07.961724 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:07.961786 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.961867 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.962027 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8244,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":983467,"threads_started":2}
W20260709 12:15:08.009608 28735 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
I20260709 12:15:07.936362 28494 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.033293 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.035982 23784 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.819213 28620 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.045555 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:07.935322 28513 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:07.943884 28589 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.940336 28487 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.045312 28686 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:08.010748 28657 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:07.924695 28685 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.056938 23683 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.070498 28646 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:08.060693 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 172600us
I20260709 12:15:07.999972 28751 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:07.942179 28550 consensus_queue.cc:1059] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 12:15:08.010484 28045 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:07.918550 28549 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.068226 28678 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.928s	user 0.015s	sys 0.002s
I20260709 12:15:08.095126 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2882) took 1222 ms. Trace:
I20260709 12:15:08.082088 23784 raft_consensus.cc:2468] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:08.078009 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 656) took 1228 ms. Trace:
I20260709 12:15:08.103127 25114 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:08.103560 23934 rpcz_store.cc:276] 0709 12:15:06.849473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.397213 (+547740us) service_pool.cc:224] Handling call
0709 12:15:07.397503 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.397700 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:07.801421 (+403721us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.077607 (+276186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.077663 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:08.077717 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.077791 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.077940 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":189435,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":46,"raft.queue_time_us":104,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":13329,"thread_start_us":679024,"threads_started":2}
I20260709 12:15:08.102025 23930 rpcz_store.cc:276] 0709 12:15:06.872864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.411442 (+538578us) service_pool.cc:224] Handling call
0709 12:15:07.412045 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.412365 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:07.742615 (+330250us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.094761 (+352146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.094817 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:08.094871 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.094945 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.095067 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15389,"prepare.queue_time_us":204651,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":7779,"raft.queue_time_us":15533,"thread_start_us":666131,"threads_started":2}
I20260709 12:15:08.118643 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:08.107955 23784 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:08.015905 28688 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.056344 28711 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:08.094784 28621 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:08.083956 28407 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.119899 28664 raft_consensus.cc:2804] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:15:08.128566 28743 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:15:08.178210 28487 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.179557 28370 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:08.128530 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2883) took 1200 ms. Trace:
I20260709 12:15:08.180621 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1356) took 1346 ms. Trace:
I20260709 12:15:08.077596 28754 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:15:08.120291 28742 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 12:15:08.137104 23929 raft_consensus.cc:2468] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:08.108148 28650 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:08.140267 28747 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:15:08.152287 28740 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:15:08.156973 28685 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.123411 28681 raft_consensus.cc:3055] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:08.164407 28537 consensus_meta.cc:287] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.002s
I20260709 12:15:08.096051 28751 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.079576 28749 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:08.180493 23934 rpcz_store.cc:276] 0709 12:15:06.928124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.127633 (+1199509us) service_pool.cc:224] Handling call
0709 12:15:08.127971 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.128184 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.128220 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.128269 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:08.128305 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.128356 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.128461 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:08.233433 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2885) took 1276 ms. Trace:
I20260709 12:15:08.181036 23932 rpcz_store.cc:276] 0709 12:15:06.834343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.320809 (+486466us) service_pool.cc:224] Handling call
0709 12:15:07.321171 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.321533 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.321558 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.322074 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 12:15:07.586293 (+264219us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:07.586342 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.586417 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.180368 (+593951us) raft_consensus.cc:1685] finished
0709 12:15:08.180390 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.180549 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1946,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"thread_start_us":263740,"threads_started":1,"wal-append.queue_time_us":427442}
I20260709 12:15:08.223351 28549 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:08.162793 28645 consensus_meta.cc:287] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.001s
I20260709 12:15:08.070691 28513 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.069759 28657 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.172971 28691 consensus_meta.cc:287] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.052s	user 0.005s	sys 0.009s
I20260709 12:15:08.119633 28678 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.220018 28045 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.222496 28716 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 1.090s	user 0.007s	sys 0.004s
I20260709 12:15:08.224587 28754 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:08.225198 28664 raft_consensus.cc:697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:08.242287 28664 consensus_queue.cc:237] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.225929 28681 raft_consensus.cc:740] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:08.180055 28370 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:08.228042 23852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1482) took 1240 ms. Trace:
I20260709 12:15:08.276818 23852 rpcz_store.cc:276] 0709 12:15:06.987295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.004121 (+ 16826us) service_pool.cc:224] Handling call
0709 12:15:08.227788 (+1223667us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54003cc154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:08.227959 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353014656}
I20260709 12:15:08.164505 28741 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.279251 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2861) took 1370 ms. Trace:
I20260709 12:15:08.126916 28711 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:08.231777 28751 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:08.234982 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 301493us
I20260709 12:15:08.234948 23932 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:15:08.067441 28686 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.233824 23934 rpcz_store.cc:276] 0709 12:15:06.957037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.232616 (+1275579us) service_pool.cc:224] Handling call
0709 12:15:08.232917 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.233093 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.233116 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.233155 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:08.233199 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.233249 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.233368 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:08.237440 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.239745 23930 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:08.248863 23925 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:08.229557 28650 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.306499 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 197138us
W20260709 12:15:08.247144 28532 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.936s	user 0.007s	sys 0.006s
I20260709 12:15:08.283646 23859 raft_consensus.cc:2368] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:08.309861 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:08.280675 23781 rpcz_store.cc:276] 0709 12:15:06.909256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.201080 (+291824us) service_pool.cc:224] Handling call
0709 12:15:07.286523 (+ 85443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.292570 (+  6047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.292607 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.309638 (+ 17031us) log.cc:844] Serialized 26 byte log entry
0709 12:15:07.870348 (+560710us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:07.974849 (+104501us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.278803 (+303954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.278895 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.278952 (+    57us) raft_consensus.cc:1685] finished
0709 12:15:08.278969 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.279169 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6480,"prepare.queue_time_us":70795,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":219938,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":25175,"thread_start_us":790992,"threads_started":2,"wal-append.queue_time_us":771978}
I20260709 12:15:08.310730 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1374) took 1387 ms. Trace:
I20260709 12:15:08.311519 23900 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:08.233767 28537 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.324105 23930 rpcz_store.cc:276] 0709 12:15:06.923683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.127053 (+1203370us) service_pool.cc:224] Handling call
0709 12:15:08.232985 (+105932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.310353 (+ 77368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.310650 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:08.282714 28711 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:08.308029 28045 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.334365 23927 raft_consensus.cc:2393] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:08.337530 23932 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:15:08.339795 23749 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:08.249683 28645 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.311160 28664 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.343606 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:08.247778 28716 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.333086 28597 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:08.347319 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:08.359309 23777 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12.
I20260709 12:15:08.342644 28712 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:08.341668 28688 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.363799 23853 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:08.321949 28532 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.345549 28764 raft_consensus.cc:2764] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:08.408430 23785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 900) took 1403 ms. Trace:
I20260709 12:15:08.408965 23785 rpcz_store.cc:276] 0709 12:15:07.005412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.385823 (+380411us) service_pool.cc:224] Handling call
0709 12:15:07.386145 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.547561 (+161416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.408096 (+860535us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003705d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:08.408345 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369360640}
W20260709 12:15:08.409799 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 51695.9us
I20260709 12:15:08.346920 28768 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:08.414399 28532 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.415920 23780 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:08.417145 28773 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:15:08.418036 28678 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:08.443116 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 122600us
I20260709 12:15:08.418272 28645 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.350765 28597 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:15:08.419013 25175 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 461) took 1945 ms (client timeout 1727 ms). Trace:
I20260709 12:15:08.361811 28407 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:08.364427 28712 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:08.421277 28741 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.423020 28544 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.424285 28659 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.339437 28681 consensus_queue.cc:260] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.429407 28739 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.630s	user 0.001s	sys 0.001s
I20260709 12:15:08.432149 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:08.433442 28690 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.348501 28664 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.341361 28370 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.350085 28549 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:15:08.434809 28513 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.434922 23851 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1481) took 1447 ms. Trace:
I20260709 12:15:08.435706 28657 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.436200 28716 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.439563 28589 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:08.440311 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:08.500521 28641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.303s	user 0.020s	sys 0.040s
I20260709 12:15:08.521955 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:08.444213 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:08.450513 25175 rpcz_store.cc:269] 0709 12:15:06.473575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.473710 (+   135us) service_pool.cc:224] Handling call
0709 12:15:08.418812 (+1945102us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540017d4d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:08.418940 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705340800}
I20260709 12:15:08.486438 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:08.525617 25177 raft_consensus.cc:2393] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:08.480809 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:08.526270 25174 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:08.488047 25169 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:15:08.409744 28688 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:08.421857 28748 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
W20260709 12:15:08.449146 28771 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:15:08.497680 25171 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:08.489395 28757 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:15:08.440811 28045 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:15:08.510803 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.490337 28712 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.512490 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2884) took 1565 ms. Trace:
I20260709 12:15:08.519893 28681 raft_consensus.cc:3060] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.514441 23851 rpcz_store.cc:276] 0709 12:15:06.987030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.079576 (+ 92546us) service_pool.cc:224] Handling call
0709 12:15:07.079945 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.147801 (+ 67856us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:07.148167 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 12:15:07.594281 (+446114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:07.594342 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:07.594398 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.434672 (+840274us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b5400413cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:08.434734 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.434861 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":607,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":10603,"raft.queue_time_us":1666,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8449,"spinlock_wait_cycles":155457408,"thread_start_us":445682,"threads_started":1,"wal-append.queue_time_us":581800}
I20260709 12:15:08.498647 28544 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:08.566433 28688 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.527254 25169 pending_rounds.cc:85] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 10
I20260709 12:15:08.596004 25169 pending_rounds.cc:107] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:15:08.596356 25169 raft_consensus.cc:2889] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:08.597211 28688 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.512521 28665 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:15:08.363754 28754 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.422607 28494 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.452275 28576 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:15:08.502396 28537 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.584818 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 686) took 1070 ms. Trace:
I20260709 12:15:08.587810 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:08.556788 28641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.020s	sys 0.041s
I20260709 12:15:08.484561 28597 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:08.590925 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.590759 23929 rpcz_store.cc:276] 0709 12:15:06.947162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.202049 (+1254887us) service_pool.cc:224] Handling call
0709 12:15:08.202598 (+   549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:08.202996 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.203026 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:08.203531 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.379847 (+176316us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:08.380316 (+   469us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.380586 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.380664 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.512098 (+131434us) raft_consensus.cc:1685] finished
0709 12:15:08.512123 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.512398 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4732,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":1177,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":19801,"spinlock_wait_cycles":36608,"thread_start_us":175944,"threads_started":1,"wal-append.queue_time_us":229170}
I20260709 12:15:08.586161 28675 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.498237 28407 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:08.587085 28702 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:08.610736 23858 rpcz_store.cc:276] 0709 12:15:07.514594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.514750 (+   156us) service_pool.cc:224] Handling call
0709 12:15:07.515127 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.515520 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.515550 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.560987 (+ 45437us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.231270 (+670283us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b18000c4660. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:08.289123 (+ 57853us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:08.289934 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.290027 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.584434 (+294407us) raft_consensus.cc:1685] finished
0709 12:15:08.584456 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.584740 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27240,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3427,"spinlock_wait_cycles":31187328,"thread_start_us":57587,"threads_started":1,"wal-append.queue_time_us":775776}
I20260709 12:15:08.508540 28690 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.613929 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:08.625006 23929 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:08.498977 28741 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.618671 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:08.621913 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.626296 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.605324 28686 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.629844 23860 raft_consensus.cc:1240] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:08.647243 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.648653 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.649283 23781 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:08.650550 25114 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:08.650179 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.661324 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.662029 23781 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:08.676002 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94856.2us
I20260709 12:15:08.678452 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2860) took 1769 ms. Trace:
I20260709 12:15:08.678777 23779 rpcz_store.cc:276] 0709 12:15:06.908998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.111634 (+202636us) service_pool.cc:224] Handling call
0709 12:15:07.112038 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.338010 (+225972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.678235 (+1340225us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54001ee6d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:08.678397 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503726976}
I20260709 12:15:08.679819 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.681147 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.621318 28702 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:08.685106 28702 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.686221 28544 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.618789 28045 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:15:08.689175 28544 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.689550 28589 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.678040 28650 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.691125 28494 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.692806 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 902) took 1283 ms. Trace:
I20260709 12:15:08.693234 23776 rpcz_store.cc:276] 0709 12:15:07.409693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.419937 (+ 10244us) service_pool.cc:224] Handling call
0709 12:15:07.690012 (+270075us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.712235 (+ 22223us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:07.712595 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.072491 (+359896us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:08.072998 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.073144 (+   146us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:08.073178 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.073253 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.437093 (+363840us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b180047ab20. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:15:08.692479 (+255386us) raft_consensus.cc:1685] finished
0709 12:15:08.692596 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.692740 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4347,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2160,"raft.queue_time_us":153091,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":4667,"spinlock_wait_cycles":61019264,"thread_start_us":359567,"threads_started":1,"wal-append.queue_time_us":507144}
I20260709 12:15:08.695242 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.728621 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:08.736941 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.737936 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:08.676451 28771 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:15:08.623515 28675 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:08.739391 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:08.919112 28690 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.920063 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:08.921344 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:08.919946 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:08.806059 28696 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:08.908694 28335 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.908437 28683 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.903332 28524 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:08.807272 23925 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55680 (request call id 653) took 1958 ms (client timeout 1587 ms). Trace:
W20260709 12:15:08.932502 23925 rpcz_store.cc:269] 0709 12:15:06.848669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.889271 (+ 40602us) service_pool.cc:224] Handling call
0709 12:15:08.807074 (+1917803us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54000f91d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:08.807194 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3103920896}
W20260709 12:15:08.762633 28776 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:15:08.717118 28045 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:08.887199 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 74568.3us
I20260709 12:15:08.924772 23779 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
W20260709 12:15:08.836615 28778 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:15:08.716073 28549 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.923519 28164 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:08.895383 28711 consensus_meta.cc:287] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.001s
I20260709 12:15:08.943854 28711 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.946076 28711 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.926295 23781 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:08.923823 28550 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:08.930744 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:08.809084 28417 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:15:08.798032 28767 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 12:15:08.935815 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.956300 23779 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 10: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:08.915922 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.956851 23776 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:08.959447 23779 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 12:15:08.958065 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 661) took 1492 ms. Trace:
I20260709 12:15:08.961606 23934 rpcz_store.cc:276] 0709 12:15:07.465189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.316363 (+851174us) service_pool.cc:224] Handling call
0709 12:15:08.316848 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:08.327880 (+ 11032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.327916 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:08.328422 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.441575 (+113153us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:08.441617 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.441688 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.957804 (+516116us) raft_consensus.cc:1685] finished
0709 12:15:08.957830 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:08.957989 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3977,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6093,"thread_start_us":112835,"threads_started":1,"wal-append.queue_time_us":263925}
I20260709 12:15:08.970220 25171 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:08.936764 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:08.927836 23851 raft_consensus.cc:3060] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.971685 23898 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:08.928931 28696 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:08.974253 28690 raft_consensus.cc:2804] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:08.957362 28550 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.974813 28690 raft_consensus.cc:697] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:08.928435 28335 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.975756 28690 consensus_queue.cc:237] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.941733 28549 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.979046 28696 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.980392 28549 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.943132 28675 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:08.948108 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:08.926779 28785 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:08.983022 28785 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:08.984736 28704 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:08.984930 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:08.985747 28785 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:08.948549 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.926831 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:08.810428 28621 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:08.737630 28494 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.981542 28550 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:08.924166 28407 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:08.955551 28417 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:08.940379 28045 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:08.998522 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 61878.1us
I20260709 12:15:09.064275 23852 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:09.028872 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2865) took 2114 ms. Trace:
W20260709 12:15:09.055437 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:09.066222 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.980443 28696 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:09.070065 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.070919 23928 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:08.987846 28657 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.070317 23857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1485) took 1327 ms. Trace:
I20260709 12:15:09.075230 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.080979 23928 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:09.065686 23778 rpcz_store.cc:276] 0709 12:15:06.914338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.388887 (+474549us) service_pool.cc:224] Handling call
0709 12:15:07.453694 (+ 64807us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:07.900829 (+447135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.900866 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:07.901568 (+   702us) log.cc:844] Serialized 52 byte log entry
0709 12:15:08.232584 (+331016us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:08.232627 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.232706 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.028543 (+795837us) raft_consensus.cc:1685] finished
0709 12:15:09.028585 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.028797 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50688,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":34542,"thread_start_us":659868,"threads_started":2,"wal-append.queue_time_us":406332}
I20260709 12:15:09.083835 23778 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:15:09.078660 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.084468 28683 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:09.149480 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 450138us
I20260709 12:15:09.149055 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:09.150772 23932 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.995016 28621 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:09.150705 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 72149.7us
I20260709 12:15:08.992970 28741 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.153359 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:09.148043 23857 rpcz_store.cc:276] 0709 12:15:07.743315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.068522 (+325207us) service_pool.cc:224] Handling call
0709 12:15:08.086618 (+ 18096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.334923 (+248305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.334960 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.335006 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:08.335040 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.070081 (+735041us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400070654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:09.070139 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.070263 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570533760}
I20260709 12:15:09.166806 23851 raft_consensus.cc:2468] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:09.156879 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:09.171383 23928 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:09.023754 28721 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:09.189262 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:09.190141 23925 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.085439 28589 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:09.192142 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.079675 28636 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.153769 28675 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:09.194401 28636 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.194696 28675 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:09.153735 28683 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.078619 28620 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.147300 28478 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:09.214095 23776 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:09.214835 23781 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:09.000380 28494 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.024550 28597 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.216673 23776 raft_consensus.cc:1240] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:09.192929 28751 consensus_meta.cc:287] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.908s	user 0.011s	sys 0.001s
I20260709 12:15:09.081224 28493 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 12:15:09.193081 28537 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:09.027945 28702 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.148434 28657 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.249325 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:09.250092 23928 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:09.215346 28760 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:09.213217 28620 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:09.253898 23749 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.68:43683 timed out after 1.727s (SENT) [suppressed 17 similar messages]
I20260709 12:15:09.193776 28589 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:15:09.264441 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.267426 28760 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:09.270817 28494 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.275185 23931 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:09.218410 28478 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.282343 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2880) took 2710 ms. Trace:
I20260709 12:15:09.284984 28620 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.269006 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:09.289625 23932 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:09.293085 23933 rpcz_store.cc:276] 0709 12:15:06.571924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.572099 (+   175us) service_pool.cc:224] Handling call
0709 12:15:06.606749 (+ 34650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:06.726289 (+119540us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:07.083826 (+357537us) log.cc:844] Serialized 14 byte log entry
0709 12:15:07.511440 (+427614us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.511972 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:15:07.512110 (+   138us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:07.512142 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.281924 (+1769782us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400334ad4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:09.281975 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.282024 (+    49us) raft_consensus.cc:1685] finished
0709 12:15:09.282119 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.282273 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32302,"prepare.queue_time_us":730910,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":17916,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":21948,"spinlock_wait_cycles":233172352,"thread_start_us":759233,"threads_started":2,"wal-append.queue_time_us":782966,"wal-append.run_cpu_time_us":2644,"wal-append.run_wall_time_us":1154215}
I20260709 12:15:09.219038 28721 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:09.285856 28681 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:09.297456 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.245098 28751 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.291604 28691 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.293321 28589 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.298919 28493 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:09.311203 28681 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:09.311818 23899 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.305858 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:09.312774 28681 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:09.313261 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.313683 28681 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:09.314217 28607 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:09.314616 28681 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.316395 25173 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:09.321208 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:09.326092 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:09.327958 25172 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:09.330142 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 76261.9us
I20260709 12:15:09.331506 23749 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:09.323418 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:09.303876 28691 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.337713 28549 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:09.340538 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:09.072870 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:09.341887 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:09.348186 28702 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.349339 28711 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:09.347512 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3338) took 2443 ms. Trace:
I20260709 12:15:09.339576 23683 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:09.349992 28711 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:09.345899 28784 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.508s	user 0.003s	sys 0.000s
I20260709 12:15:09.350903 28711 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.337462 28796 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:09.351976 28796 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.308748 28760 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.334326 28407 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:09.352413 23860 rpcz_store.cc:276] 0709 12:15:06.904056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.905457 (+  1401us) service_pool.cc:224] Handling call
0709 12:15:06.953611 (+ 48154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.631778 (+1678167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.347232 (+715454us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540037f854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:09.347439 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3854467840}
W20260709 12:15:09.347494 28712 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.740s	user 0.011s	sys 0.000s
W20260709 12:15:09.349466 23780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2835) took 2442 ms (client timeout 2394 ms). Trace:
W20260709 12:15:09.353588 23780 rpcz_store.cc:269] 0709 12:15:06.907140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.907270 (+   130us) service_pool.cc:224] Handling call
0709 12:15:09.349232 (+2441962us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003a4654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:09.349369 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3298522368}
I20260709 12:15:09.346264 28751 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.352367 28796 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.358903 28621 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.353556 28712 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.361872 28712 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.362373 28621 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.362618 23853 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:09.363909 28664 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.343796 28638 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.367719 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:09.361694 28779 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.550s	user 0.005s	sys 0.001s
I20260709 12:15:09.370111 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:09.373842 28492 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.374440 28492 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.374702 28702 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:09.375543 28702 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:09.375898 28702 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:09.376742 28496 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.377511 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1484) took 1634 ms. Trace:
I20260709 12:15:09.377882 23854 rpcz_store.cc:276] 0709 12:15:07.743042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.896298 (+153256us) service_pool.cc:224] Handling call
0709 12:15:08.280518 (+384220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.280723 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.280748 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:08.280793 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:08.280830 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.280882 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.377337 (+1096455us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b540006f9d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:09.377453 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882790272}
I20260709 12:15:09.379040 28621 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:09.365612 28721 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.335150 28669 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:15:09.381255 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1370) took 2534 ms. Trace:
I20260709 12:15:09.381647 23927 rpcz_store.cc:276] 0709 12:15:06.847000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.298864 (+451864us) service_pool.cc:224] Handling call
0709 12:15:09.381055 (+2082191us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b54002026d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:09.381191 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185446656}
I20260709 12:15:09.381978 28669 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:09.382412 28669 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:09.384357 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.384819 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.385334 25173 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:09.385502 23854 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:09.386619 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.387431 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.387714 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.388423 23851 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:09.388911 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.389262 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.389932 23852 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.392401 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.393085 23853 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:09.393525 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.394248 23852 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 10: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.395344 23855 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.398358 23858 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.399930 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.401216 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.402408 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.403847 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.397644 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.400606 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.407562 23852 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:09.408591 23899 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.409034 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.409629 23860 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:09.405754 23854 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:09.412062 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:09.413816 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.416144 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:09.396553 28496 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:09.426497 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:09.425714 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 91332.1us
I20260709 12:15:09.376734 28492 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.429319 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.427569 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 904) took 2016 ms. Trace:
I20260709 12:15:09.430204 23783 rpcz_store.cc:276] 0709 12:15:07.411489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.606470 (+194981us) service_pool.cc:224] Handling call
0709 12:15:07.642339 (+ 35869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.642678 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.642707 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.643237 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:15:08.440933 (+797696us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b180047afa0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:08.501911 (+ 60978us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:08.502865 (+   954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.503089 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.361030 (+857941us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:09.427353 (+ 66323us) raft_consensus.cc:1685] finished
0709 12:15:09.427373 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.427509 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":4984,"spinlock_wait_cycles":73398400,"thread_start_us":60776,"threads_started":1,"wal-append.queue_time_us":1033481}
I20260709 12:15:09.432971 23927 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:09.398310 28751 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.389858 28621 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.436142 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.436082 23751 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:09.428040 28524 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:09.407094 28597 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.425348 28164 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:09.440237 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:09.417312 28764 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:09.376703 28702 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.428438 28712 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:09.464497 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 662) took 1999 ms. Trace:
I20260709 12:15:09.448644 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:09.438609 28496 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:09.468936 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:09.469638 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.443229 28524 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:09.439023 28756 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:09.472366 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:09.473721 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:09.461599 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:09.482156 25172 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:09.478535 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:09.479918 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:09.428332 28796 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.445711 28620 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.465338 23930 rpcz_store.cc:276] 0709 12:15:07.465437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.331650 (+866213us) service_pool.cc:224] Handling call
0709 12:15:08.332028 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.332257 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:08.344348 (+ 12091us) log.cc:844] Serialized 14 byte log entry
0709 12:15:09.341670 (+997322us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b180033e920. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:15:09.464123 (+122453us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:09.464189 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:09.464238 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.464310 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.464426 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11922,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3039,"raft.queue_time_us":100,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":3815680,"thread_start_us":114006,"threads_started":1}
I20260709 12:15:09.504510 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.439122 28751 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.504864 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.503674 23751 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:09.505175 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:09.511729 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:09.467749 28711 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:09.548363 25173 raft_consensus.cc:2368] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:09.502635 28702 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.538589 23934 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:09.501631 28597 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:09.456933 28589 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.466334 28764 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.502164 28621 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.437067 28669 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.471879 28712 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.537855 28681 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:09.539213 28707 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:09.506016 28478 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:09.478009 28493 consensus_queue.cc:1175] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0ee4ab0747914e7abc642312517b5e25" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 8 last_received_current_leader { term: 12 index: 10 } }
I20260709 12:15:09.613375 28815 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:09.631933 28801 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:15:09.632097 28815 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.633131 28815 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:09.633884 28810 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:15:09.622324 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 119544us
I20260709 12:15:09.559084 28785 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.635563 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:09.588109 28696 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.500s	user 0.004s	sys 0.007s
I20260709 12:15:09.540812 28711 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:09.636268 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 99016.1us
W20260709 12:15:09.635954 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 131026us
I20260709 12:15:09.559087 28607 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:09.633751 28589 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.611399 28809 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:09.641600 28796 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.641956 28712 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.640269 28785 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.644795 28813 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:09.611052 28721 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.645731 23751 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:09.610948 28664 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.646901 28711 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.658560 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:09.659709 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:09.660406 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:09.642380 28764 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.663354 23780 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.634931 28496 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.669242 28785 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.655081 28809 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.670013 28809 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:09.659126 28813 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:09.673386 28813 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:09.675644 28751 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.679090 28636 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.679698 28741 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:09.683106 23856 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:15:09.687119 23825 connection.cc:256] RPC call timeout handler was delayed by -1.4807059549998485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 12:15:09.664240 28707 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:09.660917 28621 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:09.699936 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.662066 28696 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.701602 28597 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:09.646102 28524 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.692060 28636 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.695430 28707 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.664167 28664 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.699098 28621 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:09.731396 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.708628 28696 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.748172 28417 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.749503 28417 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:09.752859 28621 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.753736 28621 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:09.754326 28621 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:09.755805 28804 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.321s	user 0.004s	sys 0.005s
I20260709 12:15:09.737926 28741 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:09.757650 28741 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.758273 28749 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:09.758847 28749 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.764487 28664 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.766654 28707 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.770056 28657 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.789079 28813 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:09.792413 28822 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:09.793159 28749 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.795833 28749 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.796185 28809 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.798695 28809 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:09.801069 28721 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:09.804293 28815 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:09.713692 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:09.839337 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 137427us
I20260709 12:15:09.840605 23780 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:09.843070 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2847) took 1358 ms. Trace:
I20260709 12:15:09.844799 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:09.843667 23784 rpcz_store.cc:276] 0709 12:15:08.484290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.996367 (+512077us) service_pool.cc:224] Handling call
0709 12:15:08.996714 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:08.997092 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.997122 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:09.024937 (+ 27815us) log.cc:844] Serialized 52 byte log entry
0709 12:15:09.262387 (+237450us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:09.299086 (+ 36699us) log.cc:844] Serialized 14 byte log entry
0709 12:15:09.301103 (+  2017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.301180 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.645207 (+344027us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180006fac0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6932a6 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1
0709 12:15:09.752441 (+107234us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180006fac0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c5522 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:15:09.779705 (+ 27264us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b180006fac0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6c53f3 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd 0000000000388634 0000000000388479 00007fb2aa6a883a
0709 12:15:09.842375 (+ 62670us) raft_consensus.cc:1685] finished
0709 12:15:09.842440 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.842746 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35118,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1811,"raft.queue_time_us":2932,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":68744,"spinlock_wait_cycles":14574336,"thread_start_us":237059,"threads_started":1,"wal-append.queue_time_us":277015}
W20260709 12:15:09.836321 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 179002us
I20260709 12:15:09.876504 23779 raft_consensus.cc:2368] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 13. Current term is 14.
I20260709 12:15:09.877285 28810 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:09.782320 28691 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.851907 23749 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:09.721020 28636 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.879024 28813 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:09.902478 23782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2832) took 3007 ms (client timeout 1579 ms). Trace:
W20260709 12:15:09.903177 23782 rpcz_store.cc:269] 0709 12:15:06.895071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.895239 (+   168us) service_pool.cc:224] Handling call
0709 12:15:09.901937 (+3006698us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54001eebd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:09.902232 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582580480}
I20260709 12:15:09.904901 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:09.906407 23925 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:09.879745 28721 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:15:09.911391 28815 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.907200 23778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 915) took 1321 ms. Trace:
I20260709 12:15:09.892746 23784 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:09.887723 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 193919us
I20260709 12:15:09.895323 23933 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:15:09.902346 28636 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.892912 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.896807 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.890059 28696 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.887699 28597 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.896596 28417 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.884622 28621 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:09.898177 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:09.941790 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 91373us
I20260709 12:15:09.879606 28827 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:09.940837 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 89215.3us
I20260709 12:15:09.889341 28707 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.948742 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:09.891927 28493 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.950518 23925 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:09.951155 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:09.952049 28417 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.953747 23782 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:09.895639 28825 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:09.968067 28493 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.957065 28785 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:09.980043 23859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1476) took 2979 ms (client timeout 2115 ms). Trace:
W20260709 12:15:09.980619 23859 rpcz_store.cc:269] 0709 12:15:06.968400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.968509 (+   109us) service_pool.cc:224] Handling call
0709 12:15:09.916875 (+2948366us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400024754. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:09.947346 (+ 30471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2737385728}
I20260709 12:15:09.982682 28045 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:09.983352 28809 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:09.984192 28809 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:09.986029 28809 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:09.986656 28164 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:09.987311 28164 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:09.993690 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.994940 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:09.996151 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:09.995906 28597 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:09.997288 28683 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.000582 28815 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:10.004249 28707 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:10.008216 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62571.9us
I20260709 12:15:09.974181 28493 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.014863 23854 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:09.917615 23778 rpcz_store.cc:276] 0709 12:15:08.585925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.094080 (+508155us) service_pool.cc:224] Handling call
0709 12:15:09.181593 (+ 87513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.301016 (+119423us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:09.301312 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 12:15:09.366420 (+ 65108us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:09.366494 (+    74us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:09.366545 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.906628 (+540083us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000dd4d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:09.906726 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.907005 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20668,"prepare.queue_time_us":1727,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":837,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":16482,"spinlock_wait_cycles":274069888,"thread_start_us":64038,"threads_started":1,"wal-append.queue_time_us":131858}
I20260709 12:15:10.017647 28335 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.021588 28707 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.015758 23856 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:10.028475 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.028847 23860 raft_consensus.cc:2410] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:10.029191 23930 raft_consensus.cc:1240] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:10.030301 25177 raft_consensus.cc:2410] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:10.000792 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:10.032042 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 56141.6us
I20260709 12:15:10.029143 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.033232 23749 leader_election.cc:400] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:10.033653 23749 leader_election.cc:403] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:10.032197 28721 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:10.046715 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.049803 28721 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:10.050422 28721 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.054451 23934 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:10.042169 28825 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.059188 28711 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.060503 28825 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.061592 28711 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.026335 28785 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:10.064706 28828 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 12:15:10.065160 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.075237 28597 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.075284 28417 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:10.044842 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94238.8us
I20260709 12:15:10.078433 28657 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:10.078999 28664 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:10.080073 28657 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:10.080602 28657 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.082258 28657 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.080010 28665 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:10.084483 28665 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.083052 28370 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.086066 28665 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.092037 28657 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.094703 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.095003 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2845) took 1643 ms. Trace:
I20260709 12:15:10.095553 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
W20260709 12:15:10.093250 23777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40134 (request call id 907) took 2172 ms (client timeout 1617 ms). Trace:
I20260709 12:15:10.093467 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:10.096633 23931 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1371) took 3249 ms (client timeout 2623 ms). Trace:
I20260709 12:15:10.095448 23781 rpcz_store.cc:276] 0709 12:15:08.451324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.988670 (+537346us) service_pool.cc:224] Handling call
0709 12:15:09.214662 (+225992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.215221 (+   559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.094597 (+879376us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54004263d0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:10.094823 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518894848}
I20260709 12:15:10.098994 23930 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:10.098708 28815 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.100456 28760 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.101565 28815 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.107672 28669 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:10.111944 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.113047 23925 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:10.114264 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:10.118036 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 324613us
I20260709 12:15:10.125535 23930 raft_consensus.cc:1275] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:10.125784 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.126849 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.128710 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:10.129995 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 97087.7us
I20260709 12:15:10.097044 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.130395 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.130760 28749 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:10.133818 25114 leader_election.cc:400] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 13. Current term is 14.
I20260709 12:15:10.130088 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.136780 23860 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:10.140043 23783 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:10.140174 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.141075 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.043493 28741 consensus_queue.cc:1059] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.199s
I20260709 12:15:10.141834 23860 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:10.126849 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.131161 23926 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:10.142671 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 93259.9us
W20260709 12:15:10.096349 23777 rpcz_store.cc:269] 0709 12:15:07.920355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.923073 (+  2718us) service_pool.cc:224] Handling call
0709 12:15:10.092778 (+2169705us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400425ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:10.093016 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000571136}
I20260709 12:15:10.145725 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.142609 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.148934 23929 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:10.149909 28785 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.151608 28664 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:10.152640 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.153403 23927 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:10.152438 28664 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.156138 28664 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:10.157866 25113 connection.cc:256] RPC call timeout handler was delayed by -2.0161520669998936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 12:15:10.158609 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.134227 25114 leader_election.cc:403] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:10.160715 23930 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:10.176275 23854 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
W20260709 12:15:10.145830 23931 rpcz_store.cc:269] 0709 12:15:06.847253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.074912 (+227659us) service_pool.cc:224] Handling call
0709 12:15:10.096243 (+3021331us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54000f82d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:10.096422 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5058032512}
I20260709 12:15:10.187908 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.188778 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.189687 23927 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:10.190725 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.192258 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.192986 23859 raft_consensus.cc:1813] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Handling vote request from an unknown peer 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.193629 23859 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:10.198823 28825 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.199784 28417 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.200510 28417 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.200963 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.202286 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.204695 28707 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:10.205207 28597 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:10.206719 28825 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.207538 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.207864 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.208369 23860 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:10.208709 23856 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:10.208884 28813 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:10.210820 28825 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.205243 28707 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:10.212345 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 66799.9us
I20260709 12:15:10.212623 28707 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.212977 28815 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:10.213393 28815 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.213742 28815 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.208941 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.191257 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.221483 23926 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:10.223277 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:10.225382 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51160.5us
I20260709 12:15:10.227737 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.228204 23932 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:10.216195 25177 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
W20260709 12:15:10.232051 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 98441.6us
I20260709 12:15:10.216619 28417 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.237161 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.233645 28664 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.229450 28618 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:15:10.262662 23930 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:10.271411 23898 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.274231 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.275241 23858 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:10.276423 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:10.283130 28688 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:10.287860 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57450.4us
I20260709 12:15:10.288864 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.290517 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.291404 28711 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:10.291826 28749 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:10.291991 23927 raft_consensus.cc:1275] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:15:10.292464 23927 pending_rounds.cc:85] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 9
I20260709 12:15:10.292718 23927 pending_rounds.cc:107] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:15:10.292994 23927 raft_consensus.cc:2889] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:10.271294 28576 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.296522 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 108738us
I20260709 12:15:10.231256 28721 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.300117 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.305635 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.303431 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.310366 23779 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:10.312933 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.313728 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.307142 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:10.317653 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 76641.5us
I20260709 12:15:10.318046 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.321489 23856 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:10.321803 23854 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
W20260709 12:15:10.330756 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57602.4us
I20260709 12:15:10.333549 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.300158 28657 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:15:10.323463 28825 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:10.343060 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.313728 28638 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:10.336637 28690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.004s	sys 0.016s
I20260709 12:15:10.351800 28669 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:10.314648 28664 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.318559 28576 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.361416 28721 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.361229 28164 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.362416 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.343431 28825 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.364524 23900 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:10.353374 28690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.005s	sys 0.016s
I20260709 12:15:10.342825 28749 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:10.367295 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.356612 28669 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.370922 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.373348 23900 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.373615 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.374785 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.375735 23856 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:10.375828 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:10.374891 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.377015 23859 raft_consensus.cc:2368] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12.
I20260709 12:15:10.377947 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.378633 23856 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.384847 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.357584 28417 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:10.386054 25173 raft_consensus.cc:2410] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:10.384577 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.367264 28825 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.391827 23856 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:10.397902 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.400432 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.385761 28669 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.401993 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.387480 28335 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:10.409832 28335 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.410305 28335 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.412299 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.415086 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.416297 25168 raft_consensus.cc:2410] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:15:10.417271 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:10.387766 28785 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:10.391359 28657 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:15:10.418399 25171 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:10.419127 28576 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.405524 28550 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:10.419363 28657 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:10.419834 28657 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.418319 28370 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:10.420450 28370 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.420863 28370 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.421259 25171 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 476) took 1101 ms. Trace:
I20260709 12:15:10.421020 28657 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.421648 25171 rpcz_store.cc:276] 0709 12:15:09.320171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.320867 (+   696us) service_pool.cc:224] Handling call
0709 12:15:10.421015 (+1100148us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400734454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:10.421184 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520889600}
I20260709 12:15:10.422734 28657 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:10.405870 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.406245 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.407729 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.387100 28417 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.428036 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.428586 28417 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:10.430291 28665 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:10.430730 28665 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.426111 25172 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:10.425572 25174 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:10.403997 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.440346 25170 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:10.402688 28664 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.453583 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:10.454865 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61823.5us
I20260709 12:15:10.461808 25170 raft_consensus.cc:1275] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:10.462776 28721 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:10.463276 23683 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.465590 23683 catalog_manager.cc:5697] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.404608 28781 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:10.469111 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.453894 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.419539 28707 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.463306 28721 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.453294 28749 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.455521 28813 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:10.451761 28493 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:10.487259 28702 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.861s	user 0.002s	sys 0.019s
I20260709 12:15:10.455375 28550 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:10.485915 23927 raft_consensus.cc:1240] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:10.496207 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44201.7us
I20260709 12:15:10.485828 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.469919 23683 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 11, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:10.523352 23933 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
W20260709 12:15:10.516105 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98932us
W20260709 12:15:10.527876 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 58903.9us
W20260709 12:15:10.530988 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 210812us
W20260709 12:15:10.491581 28841 log.cc:927] Time spent T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
W20260709 12:15:10.520104 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 149402us
I20260709 12:15:10.503981 28721 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:10.540647 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 69441us
I20260709 12:15:10.500563 28824 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:10.541971 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.546559 23855 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.512969 28749 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.519923 28164 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:10.558380 28840 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.093s	user 0.000s	sys 0.004s
I20260709 12:15:10.562878 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.498569 28809 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.534000 28813 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:10.493784 28665 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.564442 28665 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.564764 28813 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:10.566270 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.570129 28417 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.572536 28417 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.506801 28815 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.546159 28824 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:10.575807 28815 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.497422 28707 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.498208 28781 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.532285 28597 consensus_queue.cc:1048] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:15:10.546242 28688 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 13. Current term is 14.
I20260709 12:15:10.579205 28781 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.555138 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:10.555797 28493 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.556114 23683 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 9, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.579941 25177 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:10.580736 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.588924 23933 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 8.
I20260709 12:15:10.550534 25119 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.581924 28825 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.582882 28846 consensus_queue.cc:1059] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:10.583367 25170 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:10.563742 23927 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:10.558257 28839 log.cc:927] Time spent T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.154s	user 0.000s	sys 0.003s
I20260709 12:15:10.534228 28370 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:10.599511 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44530.7us
I20260709 12:15:10.592650 28809 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.583698 28766 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:15:10.592691 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.581853 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:10.545147 28760 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.597846 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 209430us
I20260709 12:15:10.593600 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.653350 23934 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
W20260709 12:15:10.658538 28845 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.071s	user 0.000s	sys 0.004s
I20260709 12:15:10.659483 28620 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:10.659722 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:10.619568 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 69174.3us
I20260709 12:15:10.662576 23929 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
W20260709 12:15:10.661276 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 63215.2us
W20260709 12:15:10.667850 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.651798 28781 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:10.669420 28781 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.670447 28781 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.672119 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.673154 28493 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.674307 23859 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:10.674860 23859 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:10.674738 28741 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:10.675853 23859 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.676644 28749 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.673226 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.677866 23856 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:10.599255 28683 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:10.680866 28683 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.681604 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.676299 28781 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.683967 23933 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:10.687093 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.665400 23931 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:10.689803 23859 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.687279 28537 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:15:10.689532 28669 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.693038 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.669260 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:10.672971 28766 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:10.693925 28766 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.693883 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.669651 28620 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.693881 23857 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:10.695212 28766 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.696461 28664 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:10.697064 28766 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.698912 28335 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.695993 23930 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:10.674376 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 92648.6us
I20260709 12:15:10.707486 23779 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:10.651887 28846 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:10.715122 23856 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:10.726650 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.727029 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.727677 25168 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.696951 28825 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.579442 28550 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.727180 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.731112 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.676291 28760 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.712524 28741 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:10.677570 28749 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:10.737478 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:10.748581 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 86719.7us
I20260709 12:15:10.749053 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.741940 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.726585 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:10.750253 23927 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:10.743943 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.732698 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.743844 25175 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:10.730016 28164 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:10.778488 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:10.738808 28741 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.790920 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:10.791954 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 67858.5us
I20260709 12:15:10.778035 28825 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:10.780611 28621 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:10.778745 28683 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.777632 28164 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.799901 23683 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 7, term changed from 8 to 10, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 10 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.776592 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.806703 25170 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:10.807489 23751 leader_election.cc:400] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:10.746733 28721 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.789870 28741 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.810061 28621 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:10.788869 28669 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:10.814646 23824 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.819252 23934 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:15:10.820513 28848 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:10.819533 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:10.735889 28824 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.832540 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.801879 28164 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.826138 23931 raft_consensus.cc:2393] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:10.816039 28621 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.843662 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:10.816713 28707 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.816867 28825 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:10.825572 28537 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:10.866454 23933 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:10.832842 28815 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:10.824981 28664 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:10.836959 28831 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:10.873802 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:10.893713 23857 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.839983 28810 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.881300 28318 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:10.881060 28815 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.884742 28665 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:10.886694 23859 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:15:10.887225 28825 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:10.889779 28664 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.839689 28620 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.891021 25171 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:10.895685 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:10.844520 28370 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.912098 28493 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:10.912774 28493 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:10.914201 25119 leader_election.cc:400] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12.
I20260709 12:15:10.914579 25119 leader_election.cc:403] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:15:10.915292 23824 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.916206 28620 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:10.916781 28620 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.917237 28620 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.915292 28493 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.900316 23859 pending_rounds.cc:85] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 7
I20260709 12:15:10.918553 23859 pending_rounds.cc:107] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:15:10.918835 23859 raft_consensus.cc:2889] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:10.835098 28846 consensus_queue.cc:1048] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:15:10.916488 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.925565 23824 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:10.926384 28832 raft_consensus.cc:2764] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:15:10.926868 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:10.927882 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 80809us
I20260709 12:15:10.928072 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:10.928793 25171 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:10.906699 28721 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:10.878832 28707 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:10.923818 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 131437us
I20260709 12:15:10.943346 25177 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:10.931980 25119 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:10.944142 23683 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.912601 28809 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:10.974625 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:10.945111 23855 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:10.916733 28781 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12.
I20260709 12:15:10.979609 28683 raft_consensus.cc:2764] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:15:10.980203 23683 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.982015 23683 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:10.983843 28857 consensus_queue.cc:1048] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:10.983299 23683 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:10.985737 23683 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 12, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:10.993367 23857 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:10.988828 23683 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 10, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:11.015039 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 83150.7us
I20260709 12:15:10.958098 28810 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.024611 28494 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:15:11.024909 23679 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 270) took 34620 ms (client timeout 15000 ms). Trace:
I20260709 12:15:11.026857 28737 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:15:11.027391 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52884.6us
I20260709 12:15:11.031852 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:11.032835 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:11.033846 23928 raft_consensus.cc:2393] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:11.034466 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:11.035194 23929 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:11.036584 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:11.039847 25173 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:11.042016 25113 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:11.050307 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.010222 28857 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:11.121888 28766 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:11.135707 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:11.150130 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 113838us
I20260709 12:15:11.168205 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.098266 28669 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.169273 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.170598 23855 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
W20260709 12:15:11.174363 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 307252us
I20260709 12:15:11.178125 25168 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:11.185820 25119 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:11.160756 23925 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:11.189229 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:11.194567 23683 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.123567 28749 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.055493 28620 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.203116 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:11.196452 23683 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.160765 28766 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:11.242878 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 57160.1us
W20260709 12:15:11.188925 28813 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.011s	sys 0.007s
I20260709 12:15:11.188053 28370 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:11.250478 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.252890 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.185596 28493 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.200860 28857 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:11.251853 23779 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:11.254812 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:11.268417 23857 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:11.261279 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:11.247928 28813 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.012s	sys 0.007s
I20260709 12:15:11.214735 28846 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:15:11.264268 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 94244.8us
I20260709 12:15:11.271831 23781 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:11.247473 28766 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.243400 28825 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.328351 25168 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
W20260709 12:15:11.330803 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:11.337499 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.246695 28855 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:11.344498 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.329974 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:11.355046 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 323288us
I20260709 12:15:11.269567 28318 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.270727 28664 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:11.252547 28494 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:11.368119 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.248864 28597 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.097s
W20260709 12:15:11.372437 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.251111 28669 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:11.025367 23679 rpcz_store.cc:269] 0709 12:14:36.404047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.404320 (+   273us) service_pool.cc:224] Handling call
0709 12:14:36.559116 (+154796us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b54000380a8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b39a4331 00007fb2b399f3fc 00007fb2b3570e84 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:11.024628 (+34465512us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:00.384856 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:00.604965 (+220109us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:00.611889 (+  6924us) write_op.cc:435] Acquired schema lock
0709 12:15:00.616952 (+  5063us) tablet.cc:662] Decoding operations
0709 12:15:00.689550 (+ 72598us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:00.689663 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:00.689836 (+   173us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:15:00.690565 (+   729us) tablet.cc:701] Row locks acquired
0709 12:15:00.690633 (+    68us) write_op.cc:260] PREPARE: finished
0709 12:15:00.690843 (+   210us) write_op.cc:270] Start()
0709 12:15:00.691001 (+   158us) write_op.cc:276] Timestamp: P: 1783599300690774 usec, L: 0
0709 12:15:00.691050 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:15:00.701236 (+ 10186us) log.cc:844] Serialized 2454 byte log entry
0709 12:15:01.503627 (+802391us) op_driver.cc:464] REPLICATION: finished
0709 12:15:01.688826 (+185199us) write_op.cc:301] APPLY: starting
0709 12:15:01.707577 (+ 18751us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:02.039748 (+332171us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:02.071794 (+ 32046us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:02.311109 (+239315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:02.433166 (+122057us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 12:15:02.461440 (+ 28274us) write_op.cc:312] APPLY: finished
0709 12:15:02.555687 (+ 94247us) log.cc:844] Serialized 65 byte log entry
0709 12:15:03.511030 (+955343us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2aa6a80ac 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b
0709 12:15:03.793815 (+282785us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:03.808387 (+ 14572us) write_op.cc:454] Released schema lock
0709 12:15:03.836426 (+ 28039us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:03.836864 (+   438us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:03.836895 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36751,"reported_tablets":48,"spinlock_wait_cycles":42252160,"child_traces":[["op",{"apply.queue_time_us":123863,"apply.run_cpu_time_us":128674,"apply.run_wall_time_us":2170324,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":365258,"prepare.run_cpu_time_us":99352,"prepare.run_wall_time_us":620332,"raft.queue_time_us":428607,"raft.run_cpu_time_us":11336,"raft.run_wall_time_us":55207,"replication_time_us":812391,"spinlock_wait_cycles":24100096,"thread_start_us":1975413,"threads_started":5,"wal-append.queue_time_us":1679434,"wal-append.run_cpu_time_us":20286,"wal-append.run_wall_time_us":2461801}]]}
I20260709 12:15:11.379850 28825 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.389091 28825 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.379307 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.273447 28493 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.404556 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:11.410425 28853 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:15:11.411954 28494 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.409351 23777 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:11.420981 28669 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.372224 28855 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:11.424743 28855 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:11.432317 28867 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:15:11.390851 28618 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:11.434830 28618 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:11.435302 28618 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:11.440791 28620 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.444566 28493 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:11.445129 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:11.446589 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.446985 28868 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:11.448004 28856 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.438s	user 0.000s	sys 0.001s
I20260709 12:15:11.450505 28810 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.469990 28815 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.473551 28760 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.447878 28870 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:11.477308 28870 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:11.477725 28870 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:11.479177 28675 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:15:11.504853 28741 consensus_meta.cc:287] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
W20260709 12:15:11.506384 28865 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:15:11.509913 28335 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:11.511577 28657 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:11.512120 28675 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:11.513326 28675 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.517324 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.519019 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.519980 23777 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:11.521844 28860 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:11.519956 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.523247 25168 raft_consensus.cc:2410] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:15:11.480465 28851 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.525198 28851 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.526827 28766 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.529620 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.530491 23781 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.532403 23783 raft_consensus.cc:2393] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.467312 28824 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:11.417375 28825 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:11.505633 28741 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.531939 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.527455 28851 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.537535 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.537928 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.539518 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.552889 23784 raft_consensus.cc:1813] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Handling vote request from an unknown peer 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:11.553496 23784 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.539747 23782 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:15:11.540409 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.540345 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.511198 28618 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.541335 28766 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.557623 23778 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:15:11.551355 23898 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.555516 23780 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:11.375522 28707 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.576450 23898 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.576284 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.584700 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.586408 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:11.587141 23933 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:11.588971 25169 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:15:11.589829 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.590451 23780 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:11.594403 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:11.595355 23926 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:11.596071 23781 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.609454 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:11.610212 23933 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:11.371644 28318 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.619504 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:11.620417 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:11.620397 23927 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:11.585531 28690 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.624888 25177 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
W20260709 12:15:11.598392 28737 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.004s	sys 0.005s
I20260709 12:15:11.591495 28597 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:11.579263 28871 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:11.586153 28825 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:11.636938 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.614771 28877 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:11.651036 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:11.627874 28737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.004s	sys 0.006s
I20260709 12:15:11.629534 28597 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:11.667320 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:11.650852 28859 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.296s	user 0.002s	sys 0.001s
I20260709 12:15:11.672698 23780 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:11.669479 28871 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:11.669492 28766 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:11.677193 28766 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:11.677678 28766 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.677449 28871 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.680636 23860 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:11.652066 28877 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:11.690121 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.686493 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:11.694623 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 200967us
I20260709 12:15:11.656118 28832 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:11.671036 28597 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.705217 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.705964 23857 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:11.638808 28690 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.719529 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:11.719933 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.684580 28825 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.720309 23931 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.725775 28618 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.726639 28690 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:11.727250 23852 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:11.731561 23931 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:11.655539 28823 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:15:11.742426 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.743324 23781 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 8.
I20260709 12:15:11.746838 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.748648 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.749959 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.750787 23853 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:11.708467 28832 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.753749 28832 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:11.759744 28866 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.227s	user 0.000s	sys 0.003s
I20260709 12:15:11.665752 28737 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:11.705215 28164 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.842s	user 0.011s	sys 0.002s
I20260709 12:15:11.764078 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.715384 28707 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:15:11.750017 28874 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.230s	user 0.002s	sys 0.000s
W20260709 12:15:11.819757 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 54769us
I20260709 12:15:11.773286 28886 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:11.789273 23931 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:11.796000 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.823166 23853 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:11.784368 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.798187 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.798883 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.798887 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.824393 23852 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:11.794947 28766 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.807080 28823 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:11.812121 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.813261 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.825793 23777 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.813274 28877 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.821942 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.825309 23859 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.842993 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.843717 23857 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:11.832324 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52835.6us
I20260709 12:15:11.853756 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:11.819514 28537 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.817955 28707 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.819921 28831 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:15:11.852965 28621 consensus_meta.cc:287] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.994s	user 0.011s	sys 0.001s
I20260709 12:15:11.869477 23781 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:11.822551 28886 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:11.852144 28688 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:11.873517 23780 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:11.873747 23853 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:11.772904 28164 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.840058 28810 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.861532 28493 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:11.887619 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.896346 28493 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.896777 28493 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.879372 28831 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.902482 28831 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.903892 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.891026 28810 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.869979 28884 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:15:11.905656 28597 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.906719 28810 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.852667 28823 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.907366 25176 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44508 (request call id 1161) took 1200 ms. Trace:
I20260709 12:15:11.892031 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.864811 28537 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.908985 28823 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:11.909955 28537 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:11.868548 28707 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:11.903824 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.911260 28756 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:11.911754 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.911911 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.881807 28741 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.907891 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.913035 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.913851 28766 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.914181 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.903703 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.914170 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.914932 23853 raft_consensus.cc:2368] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:11.915838 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.881287 28621 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:11.916487 23784 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:11.916980 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.915911 23776 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:11.917891 28621 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:11.918331 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.887943 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.912446 23780 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:11.907806 25176 rpcz_store.cc:276] 0709 12:15:10.707337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.707581 (+   244us) service_pool.cc:224] Handling call
0709 12:15:10.714376 (+  6795us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:10.721145 (+  6769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:10.721178 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:10.783545 (+ 62367us) log.cc:844] Serialized 26 byte log entry
0709 12:15:10.993629 (+210084us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:11.123163 (+129534us) log.cc:844] Serialized 14 byte log entry
0709 12:15:11.130840 (+  7677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.130934 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.493841 (+362907us) spinlock_profiling.cc:243] Waited 9.35 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:11.907012 (+413171us) raft_consensus.cc:1685] finished
0709 12:15:11.907120 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.907268 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31422,"prepare.queue_time_us":128172,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":27070,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":2929,"spinlock_wait_cycles":21511424,"thread_start_us":267002,"threads_started":2,"wal-append.queue_time_us":400434}
I20260709 12:15:11.977909 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.929816 23857 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:11.918663 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.930153 23852 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:11.960831 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.996129 23853 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:11.929646 23782 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.998371 23779 raft_consensus.cc:2393] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:11.998968 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.927480 23859 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:11.998891 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.000612 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:11.949800 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:11.910452 28318 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:12.005957 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.009076 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:12.009939 23852 raft_consensus.cc:2368] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:12.009985 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.000999 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.018708 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:11.950335 28688 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:12.019340 23852 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.027366 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.028101 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.915859 28164 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.032291 23859 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:12.032390 23783 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:12.057138 23778 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:15:12.061223 23782 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:11.911984 28756 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.066008 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.019261 28825 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:12.067739 23852 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:12.069934 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:11.968705 28870 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:11.912485 28737 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.068527 23782 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:12.068964 23783 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:12.074473 23853 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:15:12.075759 23859 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:12.083979 23859 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:12.084889 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.086737 23780 raft_consensus.cc:2410] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:15:12.087317 23852 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:12.089038 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.075683 28537 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:12.090240 23782 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.019842 28886 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.109599 28886 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:12.111968 28870 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.116058 23782 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:12.116657 23853 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:12.119252 28493 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.119777 28537 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.121660 28756 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.083802 28597 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.123705 28537 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.118049 23853 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:12.120114 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.125301 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.076102 28825 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.128320 23857 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:12.125322 23853 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:12.129662 28825 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.130810 28493 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.137234 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:12.142947 23860 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:12.143230 23777 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
W20260709 12:15:12.100220 28824 consensus_meta.cc:287] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.003s
I20260709 12:15:12.101565 23783 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:12.148010 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 252489us
I20260709 12:15:12.148073 23783 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:12.153729 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 49000.8us
W20260709 12:15:12.155845 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.156994 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.157811 23860 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 8.
I20260709 12:15:12.155417 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.159204 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.062184 28688 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.161437 23854 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:12.164320 23783 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:12.169677 23784 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:12.135869 28848 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:15:12.170755 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.171487 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.174729 23777 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:12.168465 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.149201 28756 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:12.177016 28898 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:15:12.161134 28870 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:12.196614 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:12.197136 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.178757 23782 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:15:12.164848 28657 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:15:12.164240 28576 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:12.197842 23782 pending_rounds.cc:85] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 9
I20260709 12:15:12.198266 23782 pending_rounds.cc:107] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:15:12.198529 23782 raft_consensus.cc:2889] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:12.189453 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.190961 23785 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.191257 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56656 (request call id 3395) took 1520 ms. Trace:
I20260709 12:15:12.167091 28707 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.191942 23824 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.193091 28823 raft_consensus.cc:2764] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:12.198942 23781 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.200114 23783 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.199452 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.201848 23785 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:12.205003 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.199441 23854 rpcz_store.cc:276] 0709 12:15:10.669497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.671126 (+  1629us) service_pool.cc:224] Handling call
0709 12:15:10.671413 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.163239 (+1491826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:12.187261 (+ 24022us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54002d41d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:12.187481 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3422596480}
I20260709 12:15:12.206482 23784 raft_consensus.cc:2410] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 14 because replica has last-logged OpId of term: 12 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 12.
I20260709 12:15:12.207919 23860 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:12.198112 28576 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.203750 28878 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:12.220214 23855 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:12.223922 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.228142 23931 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.228494 23777 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:12.234382 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.235366 23932 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.237432 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.238902 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:12.221666 23929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:12.241022 23929 raft_consensus.cc:606] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:12.197830 28756 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:12.248979 28883 log.cc:927] Time spent T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.566s	user 0.001s	sys 0.005s
I20260709 12:15:12.239482 23855 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:15:12.215261 28824 raft_consensus.cc:2764] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:12.240890 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:12.228317 28576 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:12.208081 28493 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:15:12.279090 25114 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:12.270715 23854 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:12.253420 28861 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.287134 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:12.285938 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 58433.7us
I20260709 12:15:12.288697 28756 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:12.273227 28823 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:12.269974 28335 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:15:12.270720 28825 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:15:12.299823 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.329653 23783 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.284180 28657 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:12.307500 23855 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:12.307906 23824 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.254035 28878 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.334017 23931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:12.334856 23931 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:12.335282 23931 raft_consensus.cc:614] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:12.345671 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:12.340528 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:12.352262 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.355852 23854 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:12.357287 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:12.358208 25173 raft_consensus.cc:2393] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.359480 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:12.360301 25176 raft_consensus.cc:2393] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:12.360734 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:12.331816 28669 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.360245 25168 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:12.345014 28824 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.369459 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:12.369511 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.308022 28857 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:15:12.351071 28870 raft_consensus.cc:2764] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:12.370854 23783 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.385869 28877 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:12.387053 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:12.387918 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:12.295660 28861 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:12.391064 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 50615.6us
W20260709 12:15:12.370651 28907 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.143s	user 0.001s	sys 0.001s
I20260709 12:15:12.395217 23785 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:12.399785 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.401966 23751 leader_election.cc:400] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:12.402310 23751 leader_election.cc:403] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:15:12.400707 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:12.407888 23779 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:15:12.404868 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.408360 23779 pending_rounds.cc:85] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:15:12.408669 23779 pending_rounds.cc:107] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:15:12.408931 23779 raft_consensus.cc:2889] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:12.405915 23751 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.387338 28823 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:12.408752 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.414505 28870 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:12.419512 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.396306 28831 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:12.415485 23783 raft_consensus.cc:2376] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13. Re-sending same reply.
I20260709 12:15:12.385233 28824 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.370980 28669 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:12.418119 23825 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.579s (SENT)
I20260709 12:15:12.449530 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.407516 28877 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:12.458738 28335 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:12.410712 28756 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:12.462061 23751 leader_election.cc:400] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:12.462438 23751 leader_election.cc:403] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:15:12.458446 23785 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:12.397881 28886 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:15:12.465790 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.466485 23783 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:12.467034 23751 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.422652 28809 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:12.471876 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:12.424400 28831 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:12.476747 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.477663 23929 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:12.452834 28707 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.486519 23858 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:15:12.486577 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68503.4us
I20260709 12:15:12.483417 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.484081 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.488467 23783 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 18.
W20260709 12:15:12.473215 28894 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.408s	user 0.003s	sys 0.000s
I20260709 12:15:12.501065 23853 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:12.448253 28370 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:12.504448 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.505921 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.422565 28823 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.514928 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.468475 28810 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:12.469693 28537 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:12.510440 23932 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.510720 23925 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:12.506592 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.510991 23934 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:12.459312 28335 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.482852 28831 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:12.525724 23927 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:12.508606 28815 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:12.537683 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:12.542363 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 75423us
I20260709 12:15:12.503918 28809 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:12.554453 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.522890 28756 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:12.522409 28915 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.540520 28318 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.459306 28877 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.615922 28318 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.533917 28810 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:12.574539 28915 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:12.619046 28823 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.576728 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.620272 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2921) took 1037 ms. Trace:
I20260709 12:15:12.562937 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:12.563122 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:12.619355 28810 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.621685 28915 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.620612 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.622553 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:12.623337 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.624200 23785 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:12.578042 28669 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.580219 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.580910 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.581153 25173 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:12.581632 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:12.582320 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:12.583346 28916 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:15:12.584122 28913 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:15:12.586251 23926 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:12.587723 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.599925 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:12.607743 25171 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:12.609169 28815 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:12.610555 28861 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:12.612004 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:12.697181 25176 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.549336 28537 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.618295 28318 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:12.625640 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:12.632473 23934 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:12.633915 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.659914 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:12.572007 28908 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:15:12.620702 23930 rpcz_store.cc:276] 0709 12:15:11.582588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.584091 (+  1503us) service_pool.cc:224] Handling call
0709 12:15:11.614831 (+ 30740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:11.625883 (+ 11052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:11.625918 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:11.645634 (+ 19716us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.989181 (+343547us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:11.989241 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.989348 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:12.394904 (+405556us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:12.619947 (+225043us) raft_consensus.cc:1685] finished
0709 12:15:12.619973 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:12.620197 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3196,"raft.queue_time_us":4948,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":10240,"spinlock_wait_cycles":8345856,"thread_start_us":315695,"threads_started":1,"wal-append.queue_time_us":340844}
I20260709 12:15:12.666801 23927 raft_consensus.cc:2368] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
W20260709 12:15:12.667996 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68167us
I20260709 12:15:12.674350 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:12.702689 23928 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:12.703269 23785 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.700297 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.633827 28335 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.560492 28855 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.599957 28878 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.706979 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.749821 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.750337 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.750715 23785 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.688980 28766 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.696421 28669 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:12.752511 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.763782 23934 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:12.753436 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.690600 28760 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:12.708628 28925 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.074s	user 0.001s	sys 0.005s
W20260709 12:15:12.693022 28920 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:15:12.730075 28912 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.328s	user 0.000s	sys 0.006s
I20260709 12:15:12.764722 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:12.768728 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 293664us
I20260709 12:15:12.756237 28886 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:12.768468 23932 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:12.773533 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:12.771852 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:12.815572 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 414898us
I20260709 12:15:12.694648 28737 consensus_queue.cc:1059] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:15:12.821848 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:12.768167 28669 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:12.600131 28809 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.845983 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.847636 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.850505 23852 raft_consensus.cc:2393] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:12.823886 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 150662us
W20260709 12:15:12.824728 23778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.012s	sys 0.009s
I20260709 12:15:12.827620 23934 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:12.852370 23778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.013s	sys 0.009s
W20260709 12:15:12.845989 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 220440us
I20260709 12:15:12.856227 23929 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:12.777957 23927 raft_consensus.cc:1240] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:12.828574 23930 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:12.778860 28335 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:15:12.830235 23779 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:12.820196 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 276293us
I20260709 12:15:12.755748 28824 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:12.828385 23926 raft_consensus.cc:1240] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:12.829824 28848 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:12.833317 28878 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.753041 28855 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.855098 23824 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:12.858909 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.831681 28370 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.860070 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:12.863168 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:12.907799 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52825.8us
I20260709 12:15:12.908596 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:12.866835 23781 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.867516 23930 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:15:12.779664 28597 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:15:12.853341 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:12.767637 28766 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.843143 28756 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:12.779559 28760 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.847252 28669 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:12.915941 23934 raft_consensus.cc:2393] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 11: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:12.904217 28912 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:12.904734 28848 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:12.910718 28920 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:12.914332 28370 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.934805 28537 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.915643 28737 raft_consensus.cc:2764] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:15:12.933738 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:12.906005 28688 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:15:12.931093 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 88004us
I20260709 12:15:12.877418 28861 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:15:12.922816 28915 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.917681 28928 consensus_queue.cc:1059] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
W20260709 12:15:12.978168 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.969728 28760 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:12.971474 28855 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:12.971700 28756 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.940176 28848 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.967448 28933 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:15:12.978982 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:12.961370 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 189597us
I20260709 12:15:12.907457 28878 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.982622 25176 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:12.981946 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66702.9us
I20260709 12:15:12.989547 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.001804 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.005839 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:12.989275 28824 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:13.007131 28760 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:13.013445 28370 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.014680 28760 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.015188 28760 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:13.015789 28928 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:13.016247 28928 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:13.016913 28937 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:12.960979 28926 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:15:13.017426 28937 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.021399 28937 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:13.018463 28950 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.019084 28926 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:13.023447 28926 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:13.020150 28938 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:15:13.020355 28824 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:15:13.020763 28737 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:13.026644 28737 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:13.028031 28737 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.016985 23899 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.042877 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:13.050969 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:13.020823 28929 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:15:12.971746 28688 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:12.987250 28915 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.206969 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1529) took 1236 ms. Trace:
I20260709 12:15:13.185688 28877 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:13.214895 28927 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:15:12.938556 28943 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:15:12.981626 28935 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.193s
I20260709 12:15:12.964262 28931 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:13.302167 23749 connection.cc:256] RPC call timeout handler was delayed by -1.2861052299999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24750000000000003s for a response
W20260709 12:15:13.304507 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 260590us
I20260709 12:15:13.309711 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.335423 25176 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:13.317320 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.313820 23853 rpcz_store.cc:276] 0709 12:15:11.970835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.129856 (+159021us) service_pool.cc:224] Handling call
0709 12:15:12.131353 (+  1497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.509794 (+378441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:13.206625 (+696831us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400024754. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:13.206885 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849761152}
W20260709 12:15:13.322453 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 462654us
W20260709 12:15:13.326925 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 312645us
I20260709 12:15:13.338716 23899 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.339332 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.329854 23781 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:15:13.308620 28576 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.025s	user 0.010s	sys 0.010s
W20260709 12:15:13.318337 28831 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.753s	user 0.015s	sys 0.007s
I20260709 12:15:13.345458 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.346025 28930 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:13.346535 28882 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:13.349153 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.364964 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.304593 28657 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:13.314080 28318 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.382787 23782 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:13.355768 28335 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.393394 28318 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.394179 28335 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.396888 28335 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.397656 28657 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.398695 28919 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:13.400118 28657 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.400486 28550 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:13.401235 28934 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:13.402040 28934 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:13.403198 28934 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:13.407649 28885 raft_consensus.cc:993] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:13.408519 28885 raft_consensus.cc:1081] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:13.410480 28878 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.392370 28947 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:13.397038 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.391782 28944 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.350847 28688 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.347564 28766 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.348382 28943 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:13.355998 28931 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.367462 28886 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:13.419750 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43794.9us
W20260709 12:15:13.420809 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 415012us
I20260709 12:15:13.374377 28877 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:13.463889 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:13.463228 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.358487 28760 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:13.469337 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 123944us
I20260709 12:15:13.382409 28882 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:13.403538 28934 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:13.418915 28935 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.473585 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.384182 28669 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.396932 28707 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:15:13.380586 28576 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.492486 23933 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:15:13.464278 28851 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.479837 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.380420 28831 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.484970 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:13.461547 28943 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:13.500921 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:13.503065 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 192391us
I20260709 12:15:13.473094 28318 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.474725 28877 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:13.477680 28810 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:13.512162 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.513183 25169 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 10: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:13.481263 28823 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:13.351439 28928 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.515368 23851 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:13.467296 28886 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.425329 28944 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.417922 28930 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.424813 28947 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.465157 28931 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.503145 28851 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:13.540840 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:13.538808 28937 consensus_meta.cc:287] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
I20260709 12:15:13.545135 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.545855 23782 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:13.555217 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.556608 23860 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:13.560554 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.562407 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.568982 28955 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:13.569334 23859 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:13.532485 28943 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.530795 28882 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:13.569320 28960 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:15:13.572016 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.502210 28935 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:13.587201 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 47909us
I20260709 12:15:13.592571 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.593500 25176 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:13.587368 23860 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:15:13.598834 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:13.571923 28953 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.071s	user 0.005s	sys 0.000s
I20260709 12:15:13.608080 23856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:13.609007 23856 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:13.609390 23856 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.612128 25171 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:13.613492 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.618846 28947 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.581341 28576 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:13.622177 28848 consensus_meta.cc:287] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
I20260709 12:15:13.622763 28848 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.625092 28851 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:13.625438 28931 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.630285 28831 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.632067 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.632822 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.633091 28915 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.634439 28886 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.571255 28937 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.559311 28928 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.635730 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:13.636889 28848 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.631263 23851 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:13.639359 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.579702 28956 raft_consensus.cc:2764] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:13.656358 23925 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:13.658955 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.667255 25176 raft_consensus.cc:1813] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Handling vote request from an unknown peer 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.646667 23856 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.520705 28934 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.668481 25176 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:13.670028 28955 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:13.695091 23825 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.829s (SENT)
W20260709 12:15:13.666458 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 196272us
I20260709 12:15:13.643154 28882 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:13.686004 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.724018 23856 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:13.724045 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58924.6us
I20260709 12:15:13.732038 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.687696 28825 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.637629 28877 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:13.757591 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 67317.1us
W20260709 12:15:13.758188 23825 leader_election.cc:336] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.852s (SENT)
I20260709 12:15:13.738967 28760 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.751118 28823 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:13.767213 28923 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.924s	user 0.000s	sys 0.004s
I20260709 12:15:13.759709 28934 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:13.785100 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.786653 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.787949 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:13.790638 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:13.793771 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:13.796293 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64337.8us
I20260709 12:15:13.727250 28963 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:15:13.757494 28825 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.806052 28971 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:13.812171 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.546s (SENT)
I20260709 12:15:13.812719 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.789206 28871 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:13.821748 28621 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:13.822286 28621 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:13.823853 28980 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:13.823983 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.824681 28980 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.825186 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:13.825608 28937 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.826314 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:13.824575 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.828306 23859 raft_consensus.cc:1813] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Handling vote request from an unknown peer 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:13.796722 28741 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:13.825832 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.831167 23859 raft_consensus.cc:2410] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:13.831892 23856 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:13.832681 23859 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:13.832887 23856 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:13.823060 28621 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.837080 28977 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:13.837636 28977 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:13.838037 28977 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:13.839084 28935 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.839799 28944 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.840205 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.833690 28871 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.844595 28930 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.845400 28930 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:13.845866 28930 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:13.847065 28871 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.846800 28930 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.849020 28825 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.848784 28823 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.812938 28962 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:13.851966 28968 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:13.816818 28955 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.853888 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:13.879304 23781 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:13.821975 28984 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:13.877516 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.880249 28810 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.887869 23777 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:13.825213 28980 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.827361 28937 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:15:13.889493 28962 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:13.933280 23899 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.607s (SENT)
I20260709 12:15:13.933841 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.937098 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.939698 23857 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:13.942761 23899 leader_election.cc:336] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.650s (SENT)
I20260709 12:15:13.943264 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:13.937465 28983 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.054s	user 0.000s	sys 0.003s
W20260709 12:15:13.949757 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.954190 28756 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.972800 28756 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.956189 23929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 694) took 1203 ms. Trace:
I20260709 12:15:13.957703 28980 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.904520 28657 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:13.958979 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.978866 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:13.958465 28926 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:13.959231 28962 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:13.902055 28810 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.960598 28963 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:13.961154 28885 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:13.966405 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.984529 28756 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:13.985270 23899 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.718s (SENT)
I20260709 12:15:13.985769 23899 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:13.966642 28848 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.984467 23928 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1410) took 1400 ms. Trace:
I20260709 12:15:13.967634 25114 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:13.974798 23857 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:13.968158 28576 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:13.968812 28937 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:13.968420 28977 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:13.968765 28669 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:13.971248 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:13.974339 23929 rpcz_store.cc:276] 0709 12:15:12.752273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.864659 (+112386us) service_pool.cc:224] Handling call
0709 12:15:12.907992 (+ 43333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.954289 (+ 46297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:12.954326 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:12.954379 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:12.954413 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:13.955621 (+1001208us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b54001a7cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:13.955669 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:13.956032 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100946304}
W20260709 12:15:13.975889 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 172696us
I20260709 12:15:13.982751 23777 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:13.986600 23928 rpcz_store.cc:276] 0709 12:15:12.584350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.584519 (+   169us) service_pool.cc:224] Handling call
0709 12:15:12.702517 (+117998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:12.707575 (+  5058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:13.984282 (+1276707us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54002b7850. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b18ad 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:13.984407 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270585216}
I20260709 12:15:13.995837 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:14.000296 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:14.000938 23926 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:14.001225 23926 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:14.006922 23858 raft_consensus.cc:2410] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:13.999491 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:14.008350 23934 raft_consensus.cc:2368] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:15:13.984921 28810 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.009232 23928 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:14.011782 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:14.010322 28926 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:14.011811 23899 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.761s (SENT)
I20260709 12:15:14.017530 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.018560 28669 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:13.992786 28937 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.019785 28937 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.021014 28937 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:14.021459 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.006981 28576 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.026170 28766 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:14.026636 28766 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.026964 28766 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:13.985107 28657 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.031040 28657 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.981419 28944 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.034106 28944 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:14.042279 23783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43422 (request call id 2894) took 1518 ms (client timeout 1829 ms). Trace:
W20260709 12:15:14.042755 23783 rpcz_store.cc:269] 0709 12:15:12.524304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.524719 (+   415us) service_pool.cc:224] Handling call
0709 12:15:14.042029 (+1517310us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54003705d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:14.042166 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743742208}
I20260709 12:15:14.013211 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.015012 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.050390 23932 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:14.015758 23926 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.016323 28977 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.016575 28810 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.991470 28597 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:14.054342 23926 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.049746 28882 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:14.064476 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:14.051376 28318 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:14.049721 28944 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:13.993968 28848 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.058189 28657 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:14.078228 23927 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:14.078732 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.080305 23926 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:14.081586 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.084367 23824 leader_election.cc:400] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:15:14.084767 23824 leader_election.cc:403] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:15:14.085824 23928 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:14.067723 28882 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.090564 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:14.071545 28318 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.089807 28823 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 9. Current term is 10.
I20260709 12:15:14.110119 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:14.112740 23825 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.928s (SENT)
I20260709 12:15:14.093729 28882 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.097982 28318 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.079898 28934 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:15:14.132032 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 124465us
W20260709 12:15:14.131923 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 330895us
I20260709 12:15:14.133391 23749 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:14.096696 28984 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:15:14.114869 28576 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:14.152341 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.164513 23749 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.116355 28998 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:14.166491 25114 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:15:14.167725 23899 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.867s (SENT)
I20260709 12:15:14.168787 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.170149 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:15:14.171948 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 162231us
I20260709 12:15:14.137274 28823 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:15:14.178104 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.178825 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.179447 23781 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:15:14.169063 28926 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:14.187361 28926 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.188640 28741 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:14.188490 28926 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.189213 28741 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.190925 28965 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:14.168479 28998 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:14.174448 28815 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:15:14.171137 28930 consensus_queue.cc:1059] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:15:14.176075 28756 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.174165 28990 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.191408 28965 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.197072 28965 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.197908 28766 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.191411 28741 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.192920 28998 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:14.195461 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.209123 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
W20260709 12:15:14.175541 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.896s (SENT)
I20260709 12:15:14.210507 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:14.211647 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:14.212289 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:14.212834 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.213112 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:14.213413 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.215452 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.220284 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:14.224400 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48885.8us
I20260709 12:15:14.224730 23856 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:15:14.225570 23899 leader_election.cc:336] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.630s (SENT)
I20260709 12:15:14.168087 28810 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:14.227994 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:14.229053 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.230679 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.231778 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:14.233426 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 61218.6us
I20260709 12:15:14.234198 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.234874 23856 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:14.236819 23853 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:14.238977 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.242285 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.245499 28851 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:14.245499 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.195895 28756 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:14.247956 28756 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.247812 23856 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:14.249490 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.252295 23859 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:14.193189 28621 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:14.186653 28848 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:14.193847 28815 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:14.258359 28823 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:14.262605 28815 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.263788 28815 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.196210 28930 raft_consensus.cc:3055] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:14.266067 28930 raft_consensus.cc:740] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.266983 28930 consensus_queue.cc:260] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.268291 28930 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.271252 28967 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:14.272013 28810 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.273046 28810 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.274930 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.219498 28576 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:14.276542 28987 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:14.277021 28576 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.262688 28823 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:14.279143 28576 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.237545 28928 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:15:14.281219 28928 consensus_queue.cc:1243] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 438159d8de3c4027bdcf57eb81dedf86 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:15:14.281751 28928 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:14.282099 28928 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:14.277202 28597 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:14.285784 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:14.286947 23855 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 14. (index mismatch)
W20260709 12:15:14.285476 23825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.607s (SENT) [suppressed 44 similar messages]
I20260709 12:15:14.303625 28597 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:14.306123 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 72032.7us
I20260709 12:15:14.306924 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.308081 23749 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:14.321745 23825 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.046s (SENT)
I20260709 12:15:14.324107 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:14.324862 25177 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:14.326924 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:14.333544 23825 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.149s (SENT)
I20260709 12:15:14.336441 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:14.337908 23825 leader_election.cc:336] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.208s (SENT)
I20260709 12:15:14.340065 28998 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.341984 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:14.342592 28998 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.308282 28164 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:14.303979 28848 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:14.277256 28958 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:15:14.325654 28766 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:14.307544 28597 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.322685 28987 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.349464 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.351482 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.283396 28928 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.332909 28990 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:14.355405 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.359491 23853 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:14.348156 28335 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:14.348948 28848 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.346943 28164 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:14.373097 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.340626 28318 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.370641 28164 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:14.359719 29004 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:14.370031 28335 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:14.361677 28990 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:14.369863 28848 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:14.375674 28987 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:14.377013 29003 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.380648 28926 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.385382 28928 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.389353 28335 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:14.418906 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.388643 28990 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.424942 28318 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:14.426859 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 52557.6us
I20260709 12:15:14.430121 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.386205 28882 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.433669 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.417186 28970 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:15:14.452174 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:14.452468 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:14.453068 25176 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:14.438112 23899 leader_election.cc:336] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.987s (SENT)
I20260709 12:15:14.416719 28965 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.433039 28958 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.475188 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.464151 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:14.464860 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:14.465323 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:14.465644 29006 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:15:14.496202 25173 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.471038 25176 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:14.435793 28926 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:14.471926 28943 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.473086 23931 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:14.501295 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.502482 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.502249 28958 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.473615 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:14.503895 23929 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 10: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:14.504540 28958 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:14.473817 28966 log.cc:927] Time spent T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.715s	user 0.005s	sys 0.001s
I20260709 12:15:14.474841 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.474854 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.477537 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.507180 23925 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:14.478261 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.478631 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.418092 29003 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.480067 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:14.514810 23926 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.463091 28318 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.480336 29016 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.518726 29016 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:14.484453 28992 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
W20260709 12:15:14.488298 29007 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:15:14.489118 29008 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:15:14.506286 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:14.506345 23934 raft_consensus.cc:2410] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:14.508934 23927 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:14.532687 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:14.506641 23928 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:14.516374 23930 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:14.516855 28958 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:15:14.478189 28996 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.342s	user 0.001s	sys 0.000s
I20260709 12:15:14.518586 28318 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:14.522976 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 89364.3us
I20260709 12:15:14.533368 23929 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:14.524492 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.534389 25168 raft_consensus.cc:2368] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12.
W20260709 12:15:14.542240 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:14.585565 23899 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.002s (SENT)
W20260709 12:15:14.596189 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59300.9us
I20260709 12:15:14.603276 29016 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.536491 28860 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:15:14.607513 28882 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.607676 23749 leader_election.cc:400] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 10 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 10. Current term is 11.
I20260709 12:15:14.609776 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.610105 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.610868 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.611349 23926 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:14.617209 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.619443 25173 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:14.621984 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.623469 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.626951 25113 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:14.620858 28958 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:14.628876 29003 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.629350 29004 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:14.624603 28860 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:14.625236 28370 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:14.671638 23899 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.068s (SENT)
I20260709 12:15:14.635250 28809 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:14.636375 28994 log.cc:927] Time spent T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.469s	user 0.000s	sys 0.005s
I20260709 12:15:14.636737 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:14.637465 23851 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 3425) took 2245 ms (client timeout 1651 ms). Trace:
I20260709 12:15:14.639642 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:14.640797 25169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44508 (request call id 1176) took 2163 ms (client timeout 1651 ms). Trace:
I20260709 12:15:14.640426 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:14.668716 28944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.028s
I20260709 12:15:14.633975 29016 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:14.621938 28946 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:15:14.693524 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:14.648061 28965 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.697285 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.641448 29013 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
W20260709 12:15:14.699635 23851 rpcz_store.cc:269] 0709 12:15:12.392382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.392520 (+   138us) service_pool.cc:224] Handling call
0709 12:15:14.637286 (+2244766us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54004132d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:14.637392 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055650176}
I20260709 12:15:14.624336 28756 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.716069 23783 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:14.719204 28995 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.555s	user 0.000s	sys 0.005s
I20260709 12:15:14.728767 28318 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:14.707967 28576 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:14.744757 29013 consensus_queue.cc:1243] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Peer 9e0bf56d58c5480a8d088c85566558da log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:14.700361 23927 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:14.748131 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50931.8us
I20260709 12:15:14.749490 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:14.704493 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 65009.8us
I20260709 12:15:14.750973 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.752283 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.753928 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:14.645071 28882 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:14.763654 23783 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
W20260709 12:15:14.700830 25169 rpcz_store.cc:269] 0709 12:15:12.456648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.456805 (+   157us) service_pool.cc:224] Handling call
0709 12:15:14.619660 (+2162855us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400476654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:14.620045 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389560192}
I20260709 12:15:14.703295 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:14.769706 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66450.5us
I20260709 12:15:14.770721 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:14.653604 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:14.691340 28621 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:14.696761 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.648166 28870 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.693548 28926 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:15:14.705682 28974 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:14.771986 23855 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.695326 29003 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
W20260709 12:15:14.700385 28944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.016s	sys 0.031s
I20260709 12:15:14.785765 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:14.772822 28370 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:14.788363 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.789211 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.773998 28576 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:14.723188 28164 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.764366 28882 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.699369 28970 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:15:14.782222 29013 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.702440 28958 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.795298 23859 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:14.797026 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.767829 28946 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:14.797695 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.775866 28831 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:14.772701 29016 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.789896 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.802626 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.835768 23780 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
W20260709 12:15:14.786821 28979 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:15:14.781004 28809 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:14.820170 23778 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:14.796725 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:14.794373 28930 consensus_meta.cc:287] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.002s
I20260709 12:15:14.765223 28933 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:14.792856 28974 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.792752 28870 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.799945 28998 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:14.796447 28971 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.866271 28971 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.868476 28971 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.805501 28990 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.870762 28990 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.797336 28335 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.873239 28335 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.841663 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.874370 23781 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:15:14.876968 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.877600 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:14.878471 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.879063 23781 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:14.881152 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.881834 23776 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:14.842322 23784 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:14.884020 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.884858 23784 raft_consensus.cc:2410] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:15:14.885677 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.886226 23784 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:14.887046 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.819741 28882 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:14.780644 28621 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.893558 28621 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.846787 23855 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:14.851383 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.894670 23785 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:14.895706 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:14.898921 28993 log.cc:927] Time spent T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.732s	user 0.002s	sys 0.017s
I20260709 12:15:14.903992 23860 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:14.906236 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.907511 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.913074 28958 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:14.920023 29014 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.312s	user 0.000s	sys 0.003s
I20260709 12:15:14.920579 28164 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:14.922196 28848 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.000s
I20260709 12:15:14.922608 28848 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.097040 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:14.926731 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:14.925880 23825 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.478s (SENT)
I20260709 12:15:14.932170 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:14.859637 28974 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.851552 28831 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:14.860308 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.908639 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.862196 28870 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.862646 28756 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:14.832948 29013 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:14.947743 29013 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:14.865592 28998 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:14.947942 23779 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:14.948302 28998 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:14.949150 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.828588 28576 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:14.949918 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:14.950834 28997 log.cc:927] Time spent T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.833s	user 0.003s	sys 0.002s
I20260709 12:15:14.950449 28576 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.951871 28870 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:14.927889 28824 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:14.954623 28824 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.943606 28967 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:14.958666 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:14.959079 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:14.959720 23855 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:14.959659 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:14.944098 28815 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.944758 28831 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:14.940580 23778 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:14.965268 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.966032 23778 raft_consensus.cc:2393] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:14.970067 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.970502 23927 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:14.971397 23777 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:14.970708 23785 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:14.978120 25177 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:14.967384 23779 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:14.971547 23776 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:14.979773 23855 raft_consensus.cc:2468] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:14.981462 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.981539 25119 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:14.982548 23781 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:14.983924 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.984822 23784 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:14.985579 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:14.963052 28967 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.979799 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.020169 23779 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:15.014079 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.022392 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:14.949183 28848 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:15.023954 23776 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:15:14.993211 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.023222 23781 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:14.972043 28318 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:15.015525 28870 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.041682 28870 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.025969 28967 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.972541 29004 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:15:14.988108 28946 raft_consensus.cc:2804] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:15:15.054586 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 530149us
W20260709 12:15:15.052176 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.068346 28884 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:15.068815 28946 raft_consensus.cc:697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:15.069376 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 397787us
W20260709 12:15:15.070363 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.167s (SENT)
W20260709 12:15:15.071166 23899 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.173s (SENT)
I20260709 12:15:15.071501 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:15.053730 28870 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.079479 23776 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:15.082347 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.139690 23900 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:15.140026 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.140619 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:15.141470 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.142040 23783 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:15.065872 28318 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:15.073040 28956 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.016s	sys 0.015s
W20260709 12:15:15.096282 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 132529us
I20260709 12:15:15.142941 23900 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:15.080986 28882 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.109248 28550 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:15.113066 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:15.114369 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:15.147728 23927 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.147864 23925 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:15.074000 28949 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:15.126251 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.132747 25114 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:15.134516 29021 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.137825 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.142807 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.195436 23783 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:15:15.150382 28991 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 1.019s	user 0.000s	sys 0.007s
I20260709 12:15:15.151937 28335 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:15.152674 28987 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:15.156466 23927 raft_consensus.cc:2468] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:15.156064 28318 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.116091 28823 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.120275 28824 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:15.158994 28690 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:15.159690 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.165750 28882 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.166985 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:15.167042 23782 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
W20260709 12:15:15.145548 28956 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.016s	sys 0.016s
I20260709 12:15:15.204336 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.206048 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.205925 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.147785 28968 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.069859 28946 consensus_queue.cc:237] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.203974 28987 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:15.202612 28944 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:15.212644 25173 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:15.213110 23776 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:15.218139 23926 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:15.221200 23783 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:15.224718 25176 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:15.252720 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:15.253543 23683 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 169) took 12285 ms (client timeout 15000 ms). Trace:
I20260709 12:15:15.213497 25169 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:15.218911 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.225013 23854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1535) took 1742 ms. Trace:
I20260709 12:15:15.276818 23854 rpcz_store.cc:276] 0709 12:15:13.482663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.494651 (+ 11988us) service_pool.cc:224] Handling call
0709 12:15:13.504495 (+  9844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:13.758525 (+254030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:13.758555 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:13.764692 (+  6137us) log.cc:844] Serialized 26 byte log entry
0709 12:15:13.986533 (+221841us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:13.986574 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.041758 (+ 55184us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400436cd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:14.041809 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:15.028169 (+986360us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:15.224787 (+196618us) raft_consensus.cc:1685] finished
0709 12:15:15.224811 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:15.224950 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4791,"raft.queue_time_us":43920,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":209817728,"thread_start_us":402997,"threads_started":2,"wal-append.queue_time_us":298909}
I20260709 12:15:15.227869 25113 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:15.285192 23824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.064s	sys 0.079s
W20260709 12:15:15.285518 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.064s	sys 0.079s
I20260709 12:15:15.284984 23782 raft_consensus.cc:2393] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:15.285795 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 515113us
I20260709 12:15:15.288103 23926 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:15.285838 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:15.299621 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 71829.5us
I20260709 12:15:15.326575 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.326778 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.327450 23783 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:15.329630 23778 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:15.333813 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49361.2us
I20260709 12:15:15.335006 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.335662 23776 raft_consensus.cc:2376] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12. Re-sending same reply.
I20260709 12:15:15.336571 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.337184 23776 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:15.338042 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.206661 28870 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:15:15.339354 25169 raft_consensus.cc:1217] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 11.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 12:15:15.203457 29021 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.348100 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:15.342273 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.348483 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:15.349757 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.351159 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.209687 28961 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:15:15.354235 23782 raft_consensus.cc:3060] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:15.229915 28823 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:15.352752 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:15.274485 23683 rpcz_store.cc:269] 0709 12:15:02.968055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.972776 (+  4721us) service_pool.cc:224] Handling call
0709 12:15:15.253435 (+12280659us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:12.102107 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:12.111958 (+  9851us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:12.112026 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:15:12.112047 (+    21us) tablet.cc:662] Decoding operations
0709 12:15:12.113597 (+  1550us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:12.113719 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:12.113751 (+    32us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:15:12.115086 (+  1335us) tablet.cc:701] Row locks acquired
0709 12:15:12.115141 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:15:12.115365 (+   224us) write_op.cc:270] Start()
0709 12:15:12.120325 (+  4960us) write_op.cc:276] Timestamp: P: 1783599312115283 usec, L: 0
0709 12:15:12.120366 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:15:12.121330 (+   964us) log.cc:844] Serialized 5854 byte log entry
0709 12:15:12.620033 (+498703us) op_driver.cc:464] REPLICATION: finished
0709 12:15:12.963887 (+343854us) write_op.cc:301] APPLY: starting
0709 12:15:12.964003 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:12.994616 (+ 30613us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:12.994738 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:13.536175 (+541437us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:13.579641 (+ 43466us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:15:13.579721 (+    80us) write_op.cc:312] APPLY: finished
0709 12:15:13.580212 (+   491us) log.cc:844] Serialized 139 byte log entry
0709 12:15:13.733027 (+152815us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:13.763401 (+ 30374us) write_op.cc:454] Released schema lock
0709 12:15:13.770303 (+  6902us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:13.799214 (+ 28911us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:13.799277 (+    63us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3222874,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":326810,"apply.run_cpu_time_us":39709,"apply.run_wall_time_us":826124,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":339983,"prepare.run_cpu_time_us":27824,"prepare.run_wall_time_us":180750,"raft.queue_time_us":135179,"raft.run_cpu_time_us":43198,"raft.run_wall_time_us":293567,"replication_time_us":447410,"thread_start_us":762630,"threads_started":5,"wal-append.queue_time_us":293234,"wal-append.run_cpu_time_us":16078,"wal-append.run_wall_time_us":2317747}]]}
I20260709 12:15:15.344121 25169 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:15:15.212276 28690 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.364715 28690 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:15.365324 28737 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:15.365902 28737 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:15.371780 28741 raft_consensus.cc:2764] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:15:15.252171 28886 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 12:15:15.373421 28741 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.373975 28741 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.298889 28944 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:15.361689 28999 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.361073 23681 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:15.364952 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:15.356413 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.356004 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.366869 28737 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.377981 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:15.211251 28998 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.347926 28882 raft_consensus.cc:2804] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:15.388025 28810 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:15:15.391989 28882 raft_consensus.cc:697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:15.392364 28974 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:15.393069 29013 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:15.387918 28999 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.394032 29013 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:15.394425 29013 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:15.395550 28999 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:15.395272 29013 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.396806 29013 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:15.394181 29016 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:15.388489 23681 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 7 to 10, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 10 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:15.401377 23681 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 11, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.390761 23824 leader_election.cc:400] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12.
I20260709 12:15:15.404060 23824 leader_election.cc:403] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:15.404855 23824 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.406679 28961 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12.
I20260709 12:15:15.407470 28961 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:15.407989 28961 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:15.247974 28987 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.410166 28987 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.413050 29021 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:15.413892 28756 consensus_queue.cc:1175] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "05679ef5e2b74861aad2a08b6a05e276" responder_term: 14 status { last_received { term: 13 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:15.420073 28971 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.420672 28971 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.421916 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.392951 28882 consensus_queue.cc:237] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.426066 23855 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:15:15.423480 23780 raft_consensus.cc:2410] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:15:15.399302 28968 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:15.427430 23925 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:15.428144 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.429224 23852 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:15.434809 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.437243 23857 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:15.438984 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:15.438035 23776 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:15.463752 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:15.467546 23926 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:15.468324 23927 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:15.473690 23854 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:15.474380 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:15.479493 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.479656 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.479900 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.481226 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.468551 23782 raft_consensus.cc:2468] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:15.395714 28998 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.486402 23784 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:15.485478 25114 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:15.491876 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.494349 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.494627 23783 raft_consensus.cc:2376] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13. Re-sending same reply.
I20260709 12:15:15.492377 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.495929 25170 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:15.496285 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.497697 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.403362 28967 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.492715 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.445576 28971 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.523345 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.524197 23780 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:15.525161 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.525991 23780 raft_consensus.cc:2410] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:15:15.526738 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.499823 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.408895 28961 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.473335 28999 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.503259 23782 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:15.475488 28926 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.532330 28926 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.532858 28967 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:15.534235 25176 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:15.536378 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.536456 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.537664 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.537952 23780 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:15:15.538969 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.539551 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:15.542347 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.543488 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.506134 23681 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 3 to 9, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.485478 28980 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:15.551656 25114 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:15.562744 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 418342us
I20260709 12:15:15.499832 29004 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.548434 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.612823 23780 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:15.547662 23925 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:15.501103 28766 raft_consensus.cc:2804] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:15.593912 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.597218 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.597565 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.600909 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.604691 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:15.625001 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.567332 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.614485 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:15.442822 28944 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.562063 28370 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:15:15.612545 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 468660us
I20260709 12:15:15.617924 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.611449 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:15.619445 23857 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:15.625088 23778 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:15.531968 28999 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.641625 23783 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:15.643131 23776 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:15.651011 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:15.639690 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:15.663084 23780 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:15:15.663607 23780 pending_rounds.cc:85] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 5
I20260709 12:15:15.663862 23780 pending_rounds.cc:107] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:15:15.664134 23780 raft_consensus.cc:2889] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:15.650841 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:15.665596 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.672502 23681 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 7, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.674464 23778 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:15.674647 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:15.675374 23925 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.668708 23825 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.678135 25168 raft_consensus.cc:1240] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:15.679106 29022 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:15.679102 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:15.682109 23784 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:15.621644 28621 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.621582 28823 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.679754 29022 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.690438 23780 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:15.628983 28935 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:15.691864 23780 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:15:15.697594 23780 pending_rounds.cc:85] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 7
I20260709 12:15:15.697849 23780 pending_rounds.cc:107] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:15:15.698128 23780 raft_consensus.cc:2889] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:15.693277 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.698954 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:15.680433 28831 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:15.692917 23825 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.654423 28766 raft_consensus.cc:697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:15.677479 28741 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.702420 23681 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 10 to 11, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.675417 29016 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:15.654040 28926 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.701085 23859 raft_consensus.cc:2410] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:15:15.625178 29004 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.627405 28756 consensus_queue.cc:1059] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
W20260709 12:15:15.741492 29032 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:15.746755 23681 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 11, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.747912 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2965) took 1235 ms. Trace:
I20260709 12:15:15.748580 23681 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 4 to 8, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 8 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.751420 28999 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.748301 23934 rpcz_store.cc:276] 0709 12:15:14.512084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.532218 (+ 20134us) service_pool.cc:224] Handling call
0709 12:15:14.532536 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.052379 (+519843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.052413 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.052934 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.423203 (+370269us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:15.423246 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.423312 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:15.747668 (+324356us) raft_consensus.cc:1685] finished
0709 12:15:15.747694 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:15.747849 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":519404,"raft.queue_time_us":519918,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":369872,"threads_started":1,"wal-append.queue_time_us":635684}
I20260709 12:15:15.754326 23681 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:15.724514 28870 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.756103 28870 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.758392 28870 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:15.718464 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:15.699064 23928 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:15.684639 28980 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:15.764282 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.764670 23925 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:15.765560 23783 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
W20260709 12:15:15.706457 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.706116 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:15.660959 28815 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:15:15.669217 28943 consensus_queue.cc:1059] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:15:15.692328 29022 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:15.761646 23824 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:15.767882 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61835.8us
I20260709 12:15:15.722647 28968 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:15.722289 28998 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:15.777567 23681 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 6 to 9, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:15.768158 28999 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.781212 28999 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.783298 28999 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.788288 29004 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:15.697459 28823 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.789139 28968 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:15.790344 28823 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.790169 28968 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.786729 28766 consensus_queue.cc:237] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.697674 28621 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.787971 28971 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:15.751963 28935 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:15.797744 23782 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:15.806318 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.771402 28980 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.775056 28831 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:15.738722 29013 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:15:15.779276 28998 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.805864 28741 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:15.820421 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:15.807489 28823 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:15.811825 28980 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:15.814016 28956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.010s	sys 0.026s
I20260709 12:15:15.813422 28831 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.832835 28831 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:15.839318 29016 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.839825 29016 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:15.818272 28998 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:15.869925 23899 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.027s	sys 0.020s
I20260709 12:15:15.808818 28971 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:15.846025 28987 raft_consensus.cc:3055] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:15:15.898522 23899 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.028s	sys 0.020s
W20260709 12:15:15.910602 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 685090us
W20260709 12:15:15.914191 23899 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 2.493s (SENT)
I20260709 12:15:15.820333 28999 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:15:15.921787 25171 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:15.922000 28831 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.924592 28831 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.925800 23934 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:15.926934 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:15.910175 29022 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.933735 29022 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:15.834549 28823 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:15.949641 28823 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:15.860792 28956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.010s	sys 0.026s
W20260709 12:15:15.961688 23860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.011s	sys 0.026s
W20260709 12:15:15.962232 23860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.012s	sys 0.026s
I20260709 12:15:15.909998 28987 raft_consensus.cc:740] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:15.992067 25173 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:15.964187 29033 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:15.994272 29033 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:15.994616 29033 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.985419 23857 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:16.001236 28998 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.003906 28998 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.014181 29033 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:15.955072 29029 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:15:15.954509 28971 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.024879 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:15.954659 28935 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:15:16.084065 28956 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:16.049234 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 121138us
I20260709 12:15:15.963443 28974 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:15.963485 28984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.007s	sys 0.030s
W20260709 12:15:16.109252 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 453695us
I20260709 12:15:16.077651 28621 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.126137 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 709) took 1017 ms. Trace:
I20260709 12:15:16.101886 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:16.063756 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 401313us
I20260709 12:15:16.027810 29034 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:15:16.077939 28882 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:15:16.108292 28984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.007s	sys 0.031s
I20260709 12:15:16.119141 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2954) took 1115 ms. Trace:
W20260709 12:15:16.135030 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 484280us
I20260709 12:15:16.140332 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:16.142585 23858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.008s	sys 0.016s
I20260709 12:15:16.137200 23781 rpcz_store.cc:276] 0709 12:15:15.003955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.040681 (+ 36726us) service_pool.cc:224] Handling call
0709 12:15:15.040940 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.045583 (+  4643us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:15.079252 (+ 33669us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.497936 (+418684us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.498882 (+   946us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.499357 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.499483 (+   126us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:15.508063 (+  8580us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.508408 (+   345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.508499 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.118896 (+610397us) raft_consensus.cc:1685] finished
0709 12:15:16.118918 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.119077 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17430,"prepare.queue_time_us":105176,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":105,"raft.queue_time_us":44087,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":49324,"thread_start_us":323118,"threads_started":1,"wal-append.queue_time_us":546360}
I20260709 12:15:16.147770 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.123265 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.154266 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.156365 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.157040 23782 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:16.158146 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.160379 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.161121 23782 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:16.204116 23779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43438 (request call id 2952) took 1325 ms. Trace:
I20260709 12:15:16.108810 28335 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:15:16.204617 23779 rpcz_store.cc:276] 0709 12:15:14.878968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.023470 (+144502us) service_pool.cc:224] Handling call
0709 12:15:15.038476 (+ 15006us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:15:15.045973 (+  7497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.046009 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:15:15.068220 (+ 22211us) log.cc:844] Serialized 78 byte log entry
0709 12:15:15.233970 (+165750us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4800003cd8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:15:15.285869 (+ 51899us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18003acd80. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:15.693458 (+407589us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:15.693499 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.693565 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.203820 (+510255us) raft_consensus.cc:1685] finished
0709 12:15:16.203844 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.204027 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6355,"raft.run_cpu_time_us":3296,"raft.run_wall_time_us":188595,"spinlock_wait_cycles":43237888,"thread_start_us":407365,"threads_started":1,"wal-append.queue_time_us":994787}
I20260709 12:15:16.107884 28956 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.088917 28987 consensus_queue.cc:260] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:16.232689 23898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.039s
W20260709 12:15:16.243366 23898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.000s	sys 0.040s
W20260709 12:15:16.235366 23927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.011s	sys 0.028s
W20260709 12:15:16.244136 23927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.011s	sys 0.028s
I20260709 12:15:16.150715 28851 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:16.150787 28870 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:16.228720 29036 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:15:16.163431 28949 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:16.258235 23926 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.017s	sys 0.025s
I20260709 12:15:16.265769 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:16.266203 23926 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.017s	sys 0.025s
I20260709 12:15:16.259553 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:16.266860 23851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.022s	sys 0.022s
I20260709 12:15:16.267122 23925 raft_consensus.cc:2368] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
W20260709 12:15:16.267197 23851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.022s	sys 0.022s
I20260709 12:15:16.268280 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.269783 23925 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:15:16.259771 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.261376 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.261684 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.262775 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.274168 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.275152 23928 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:16.142225 28971 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
W20260709 12:15:16.251581 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 103907us
I20260709 12:15:16.234196 28884 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:16.137785 23932 rpcz_store.cc:276] 0709 12:15:15.109010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.111021 (+  2011us) service_pool.cc:224] Handling call
0709 12:15:15.111357 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.111777 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.111807 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.112314 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.237190 (+124876us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b48002601d8. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b26dbaa1 00007fb2b26d2b37 00007fb2b26d29ca 00007fb2b27ca26f 00007fb2b27c9f37 00007fb2b27de55c 00007fb2b27de508 00007fb2b27de488 00007fb2b27de443 00007fb2b27dd6ec 00007fb2b1707a96
0709 12:15:15.560553 (+323363us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:16.109038 (+548485us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.117050 (+  8012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.117136 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.117178 (+    42us) raft_consensus.cc:1685] finished
0709 12:15:16.125438 (+  8260us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.126079 (+   641us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8161,"raft.run_cpu_time_us":1654,"raft.run_wall_time_us":130371,"spinlock_wait_cycles":32152448,"thread_start_us":943249,"threads_started":2,"wal-append.queue_time_us":731729}
I20260709 12:15:16.281080 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:16.143823 28824 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:15:16.282415 23778 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:16.283324 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.293473 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:16.295485 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.252715 28576 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:16.254384 28851 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:16.310078 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.312011 23929 raft_consensus.cc:2368] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
W20260709 12:15:16.303875 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 481390us
I20260709 12:15:16.304179 23928 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:16.318557 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.320179 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:16.321636 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.322803 23784 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
W20260709 12:15:16.327267 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.329152 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46546.2us
W20260709 12:15:16.314643 25172 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.005s	sys 0.042s
I20260709 12:15:16.313006 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:16.297760 29035 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:15:16.333173 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.285569 28987 raft_consensus.cc:3060] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.332424 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.332471 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:16.338680 23779 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:16.340037 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:16.311635 28943 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.015s	sys 0.015s
I20260709 12:15:16.341768 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.333568 28823 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:16.316116 28851 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.281612 28949 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:16.353112 23777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.013s	sys 0.018s
I20260709 12:15:16.355060 23933 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:16.355346 23777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.013s	sys 0.018s
W20260709 12:15:16.353168 23780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.016s	sys 0.012s
I20260709 12:15:16.338171 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.350401 23928 raft_consensus.cc:1240] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:16.359755 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.362514 23779 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:16.356134 23780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.017s	sys 0.012s
W20260709 12:15:16.368922 29045 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:15:16.330117 28576 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.343223 29022 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:16.354707 28766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.011s	sys 0.022s
I20260709 12:15:16.327782 28886 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:15:16.349990 28335 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:16.357028 29038 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:15:16.378690 23932 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:16.352205 29053 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:15:16.370867 29046 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:15:16.382085 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.385995 23824 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:16.357254 29034 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:16.411861 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.021s	sys 0.059s
W20260709 12:15:16.378197 28766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.011s	sys 0.022s
W20260709 12:15:16.381105 28926 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.015s	sys 0.015s
I20260709 12:15:16.373729 28741 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:16.414630 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.021s	sys 0.059s
W20260709 12:15:16.417274 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 755378us
W20260709 12:15:16.420262 23783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.016s	sys 0.022s
W20260709 12:15:16.420876 23783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.016s	sys 0.022s
I20260709 12:15:16.422199 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:16.422739 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.423556 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.424454 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.377946 28947 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.372222 28943 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:16.427038 23782 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:16.439308 23782 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:16.422111 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.440225 23782 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.441372 23782 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:16.441685 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.443197 23928 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:16.435212 23932 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9.
I20260709 12:15:16.436290 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.386792 28335 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:16.452124 23784 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:16.405848 28884 raft_consensus.cc:2764] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:16.407763 29057 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:16.391533 29056 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:16.457109 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.407528 29053 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:16.485478 25171 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:16.504848 25172 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:16.453441 29023 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.509644 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.510777 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.511384 23933 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:15:16.476789 28335 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.518667 25174 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:16.522030 25176 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:16.508834 29047 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
W20260709 12:15:16.546075 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 405868us
I20260709 12:15:16.494854 28831 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:16.510257 28690 consensus_meta.cc:287] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 1.142s	user 0.011s	sys 0.003s
I20260709 12:15:16.428246 29034 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:16.536165 29044 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:15:16.536880 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.501538 29033 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:16.513358 29043 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
I20260709 12:15:16.481415 23930 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:16.438716 29022 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:16.550768 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.556867 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.621950 23928 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:15:16.459512 28947 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.519935 28956 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.518864 29066 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.644599 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1560) took 1202 ms. Trace:
I20260709 12:15:16.525061 29023 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.566253 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.602173 23778 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:16.605666 23933 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:16.509240 28943 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:16.564925 29054 log.cc:927] Time spent T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 12:15:16.630995 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 80331.2us
I20260709 12:15:16.671775 23900 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:16.646837 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 313713us
I20260709 12:15:16.674752 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.675971 23899 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.661824 23853 rpcz_store.cc:276] 0709 12:15:15.442266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.463542 (+ 21276us) service_pool.cc:224] Handling call
0709 12:15:15.463991 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:15.464402 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.464430 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:15.475889 (+ 11459us) log.cc:844] Serialized 52 byte log entry
0709 12:15:15.986368 (+510479us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:16.063314 (+ 76946us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.063618 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.063696 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.644297 (+580601us) raft_consensus.cc:1685] finished
0709 12:15:16.644325 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.644499 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1589,"prepare.queue_time_us":46270,"prepare.run_wall_time_us":50,"raft.queue_time_us":2375,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":7466,"spinlock_wait_cycles":5104384,"thread_start_us":467317,"threads_started":1,"wal-append.queue_time_us":704547}
I20260709 12:15:16.678241 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:16.679073 23857 raft_consensus.cc:2393] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
W20260709 12:15:16.681427 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 272582us
I20260709 12:15:16.560053 28690 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.559793 28831 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.685109 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.692803 23851 raft_consensus.cc:1240] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:16.695322 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 191719us
I20260709 12:15:16.691962 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.696583 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.696946 25172 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 9.
I20260709 12:15:16.659224 29022 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:16.691130 28980 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.839s	user 0.009s	sys 0.003s
I20260709 12:15:16.712610 23929 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:15:16.683221 28884 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:16.691494 28968 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:15:16.715221 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 224285us
I20260709 12:15:16.715915 23860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 743) took 1454 ms. Trace:
I20260709 12:15:16.721004 23926 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:16.703419 28944 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:16.723248 23860 rpcz_store.cc:276] 0709 12:15:15.261213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.261365 (+   152us) service_pool.cc:224] Handling call
0709 12:15:15.277960 (+ 16595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.298545 (+ 20585us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:15.298577 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.334607 (+ 36030us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.962639 (+628032us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:15.962685 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.962762 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.715659 (+752897us) raft_consensus.cc:1685] finished
0709 12:15:16.715684 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.715852 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53543,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6912,"thread_start_us":586456,"threads_started":1,"wal-append.queue_time_us":958092}
I20260709 12:15:16.686661 28974 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:15:16.670423 29064 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:15:16.734791 23749 leader_election.cc:400] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
W20260709 12:15:16.736776 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60869.4us
I20260709 12:15:16.723479 29033 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:16.673979 29066 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.720145 29055 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:16.745944 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.746737 23749 leader_election.cc:403] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:15:16.714495 28949 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:16.732004 28870 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:16.754504 23927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 2979) took 1159 ms. Trace:
I20260709 12:15:16.733723 28968 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:16.720542 28980 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.762857 28884 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:16.766211 29064 consensus_queue.cc:1243] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:15:16.774241 29068 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.172s	user 0.001s	sys 0.001s
I20260709 12:15:16.790925 28980 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.764454 28576 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.697471 29053 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.797185 29066 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.798470 29023 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.800210 28831 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.789950 29034 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.755924 29055 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:16.803284 29016 consensus_meta.cc:287] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.000s
I20260709 12:15:16.770900 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.753643 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.775050 23927 rpcz_store.cc:276] 0709 12:15:15.595161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.595298 (+   137us) service_pool.cc:224] Handling call
0709 12:15:15.595678 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.595926 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.595952 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.596403 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.244536 (+648133us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.244584 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.244658 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.754261 (+509603us) raft_consensus.cc:1685] finished
0709 12:15:16.754287 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.754437 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":559273,"threads_started":1,"wal-append.queue_time_us":694366}
W20260709 12:15:16.780248 29072 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 12:15:16.783703 29050 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:15:16.784003 28690 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:16.784507 28947 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.785815 29013 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:16.758379 29081 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:15:16.788028 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2934) took 1363 ms. Trace:
I20260709 12:15:16.787634 28968 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:16.788877 29039 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.682s	user 0.003s	sys 0.000s
I20260709 12:15:16.789297 28943 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.774032 28926 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
I20260709 12:15:16.792362 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2937) took 1278 ms. Trace:
I20260709 12:15:16.776934 28870 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.802876 29055 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.761720 29022 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:16.789991 29048 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.514s	user 0.000s	sys 0.001s
I20260709 12:15:16.808061 23928 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:16.827356 23780 rpcz_store.cc:276] 0709 12:15:15.513994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.762409 (+248415us) service_pool.cc:224] Handling call
0709 12:15:15.762745 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.776151 (+ 13406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.776182 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.776772 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.366084 (+589312us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:16.390835 (+ 24751us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.420089 (+ 29254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.420159 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.792145 (+371986us) raft_consensus.cc:1685] finished
0709 12:15:16.792166 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.792303 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57677,"prepare.queue_time_us":40236,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3083,"raft.queue_time_us":159460,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":12070,"thread_start_us":548562,"threads_started":1,"wal-append.queue_time_us":698048}
W20260709 12:15:16.854771 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 176497us
I20260709 12:15:16.854912 23853 raft_consensus.cc:1240] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:15:16.808378 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.809763 29023 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:16.804901 29016 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.810755 23858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56648 (request call id 1558) took 1370 ms. Trace:
W20260709 12:15:16.814119 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60559us
I20260709 12:15:16.815320 23777 rpcz_store.cc:276] 0709 12:15:15.424525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.741531 (+317006us) service_pool.cc:224] Handling call
0709 12:15:15.741887 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.775684 (+ 33797us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:15.842041 (+ 66357us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b18003dace0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf 00007fb2ad7cc038
0709 12:15:16.355746 (+513705us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.681801 (+326055us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.682279 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.682421 (+   142us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:16.682453 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.682522 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.787681 (+105159us) raft_consensus.cc:1685] finished
0709 12:15:16.787791 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.787958 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104717,"prepare.queue_time_us":57257,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":108130,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":3096,"spinlock_wait_cycles":21842048,"thread_start_us":785649,"threads_started":2,"wal-append.queue_time_us":340752}
W20260709 12:15:16.822257 25177 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53494 (request call id 501) took 1480 ms (client timeout 1710 ms). Trace:
W20260709 12:15:16.868803 25177 rpcz_store.cc:269] 0709 12:15:15.341523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.343144 (+  1621us) service_pool.cc:224] Handling call
0709 12:15:16.821955 (+1478811us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004be1d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:16.822156 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466342272}
I20260709 12:15:16.869416 29016 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:16.868443 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:16.870317 23776 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:16.870594 28335 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:16.872823 23925 raft_consensus.cc:1240] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:16.818360 29094 raft_consensus.cc:2764] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:15:16.855847 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:16.856951 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 48668.8us
I20260709 12:15:16.858355 28851 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.860893 29033 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:16.874490 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.879918 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:16.881569 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:16.868005 23858 rpcz_store.cc:276] 0709 12:15:15.440138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.440277 (+   139us) service_pool.cc:224] Handling call
0709 12:15:15.440848 (+   571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.441259 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.441360 (+   101us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.497613 (+ 56253us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.143096 (+645483us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:16.143144 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.143239 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.810512 (+667273us) raft_consensus.cc:1685] finished
0709 12:15:16.810540 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.810688 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2820,"raft.queue_time_us":427,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7213,"thread_start_us":509280,"threads_started":1,"wal-append.queue_time_us":885253}
I20260709 12:15:16.884846 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.886914 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:16.888659 23749 leader_election.cc:400] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
I20260709 12:15:16.889038 23749 leader_election.cc:403] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:15:16.889837 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.878803 23824 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:16.901437 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:16.879973 28987 consensus_meta.cc:287] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.004s
I20260709 12:15:16.876659 28980 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:16.891048 29077 raft_consensus.cc:2764] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:15:16.909626 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.909540 29090 consensus_queue.cc:1059] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:16.921334 23781 raft_consensus.cc:1275] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:15:16.921772 23781 pending_rounds.cc:85] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 10
I20260709 12:15:16.922004 23781 pending_rounds.cc:107] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:15:16.907756 29095 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:16.882893 29033 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:16.902510 29086 consensus_queue.cc:1059] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:15:16.909821 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.913035 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.883069 28824 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:15:16.913535 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:16.904518 29081 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:16.912120 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.915978 29064 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.890838 29023 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:16.902504 28943 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 11. Current term is 12.
I20260709 12:15:16.891268 29016 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:16.908507 29013 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:15:16.914613 28884 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:16.923532 29086 raft_consensus.cc:3055] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:16.938656 29086 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:16.930697 29096 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:15:16.932976 29089 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 12:15:16.931893 29069 log.cc:927] Time spent T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:15:16.943017 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:16.944137 23781 raft_consensus.cc:2889] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:16.939584 29086 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.944351 28824 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:16.944754 29064 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.944922 28824 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.945699 28884 consensus_queue.cc:1243] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:16.945909 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.946183 28831 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:16.947183 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.947341 29021 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:16.948065 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:16.925030 29055 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.949199 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:16.945080 29086 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:16.943302 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2939) took 1406 ms. Trace:
I20260709 12:15:16.954161 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.955385 23783 rpcz_store.cc:276] 0709 12:15:15.537346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.766810 (+229464us) service_pool.cc:224] Handling call
0709 12:15:15.767088 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:16.436098 (+669010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.436151 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.444472 (+  8321us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.685569 (+241097us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:16.713159 (+ 27590us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.713591 (+   432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.713668 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.932387 (+218719us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:16.942979 (+ 10592us) raft_consensus.cc:1685] finished
0709 12:15:16.943089 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.943242 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31848,"prepare.queue_time_us":28757,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":3274,"raft.queue_time_us":144340,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":38125,"spinlock_wait_cycles":14425728,"thread_start_us":795961,"threads_started":2,"wal-append.queue_time_us":313988}
I20260709 12:15:16.922123 28851 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:16.937330 28980 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.956671 29013 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:16.876951 28926 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:16.958573 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:16.954582 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43938.1us
I20260709 12:15:16.923640 29087 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:16.967984 29087 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:16.969022 29087 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.970686 28756 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:16.971114 28756 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:16.972940 29064 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.957589 28831 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:16.973745 28831 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:16.974160 28831 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:16.954597 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.932804 28335 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:16.951422 28886 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:16.982622 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.982045 29070 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:16.983423 29070 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:16.983911 29070 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:16.982971 29021 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:16.984752 29021 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:16.985667 29021 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.986030 29033 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:16.960332 28824 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:16.963096 28980 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:16.965361 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e60a1c6520545729d27474d8f237ce5 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:50860
W20260709 12:15:16.990347 29092 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:15:16.986200 29053 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:16.980254 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.996151 23927 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:16.968273 28815 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:15:16.992753 25177 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: [13.14-13.14]
I20260709 12:15:16.993108 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.002490 25172 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:16.993378 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.955634 28987 consensus_queue.cc:1059] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:17.003849 23929 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
I20260709 12:15:16.980711 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.006495 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:16.997483 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:16.997746 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:16.979707 23825 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.009380 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.013715 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:16.961374 28741 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 12:15:17.017776 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.990403 28980 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.016685 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.019330 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:17.023555 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:17.015262 28815 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:17.025944 28984 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:17.027485 28984 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.027927 28984 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:17.017482 28987 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:17.017634 28926 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.025377 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.019872 28737 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:17.017776 28998 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:17.017303 29022 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:15:17.030920 23931 raft_consensus.cc:2410] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:15:17.033743 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:17.034101 23825 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.025218 28980 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:17.026387 28815 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:17.030208 28987 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.032164 28737 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.044935 23776 raft_consensus.cc:3055] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:17.047994 23776 raft_consensus.cc:740] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:17.045500 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.049590 23777 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:17.049922 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.050763 23779 raft_consensus.cc:2410] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:17.033581 28886 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:17.034371 29033 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:17.048856 23776 consensus_queue.cc:260] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.057397 23776 raft_consensus.cc:3060] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.033916 28926 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.067502 28926 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.055137 28980 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:17.054360 28886 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:17.070492 28886 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:17.045394 29013 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.072834 29013 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.073206 28370 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.073856 28370 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.075243 29013 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.078105 28946 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.041916 28944 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:15:17.080278 28815 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.052124 28998 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:17.087854 23934 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:17.050103 28987 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:17.083031 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.051625 28737 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:17.111300 23930 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:17.093263 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.094656 28370 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:17.087656 28870 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.095714 28946 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.096818 28815 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:17.122658 29104 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:15:17.097595 28831 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.099093 23783 pending_rounds.cc:85] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 8
I20260709 12:15:17.125809 23783 pending_rounds.cc:107] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:15:17.126127 23783 raft_consensus.cc:2889] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:17.100121 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.101280 25176 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:17.104291 29070 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.106242 29066 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:17.107034 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.110538 23776 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 12:15:17.081501 29064 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.120240 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.133373 23778 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:15:17.120105 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:17.134658 25174 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:17.130035 25177 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:17.135251 23776 pending_rounds.cc:85] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 12
I20260709 12:15:17.135530 23776 pending_rounds.cc:107] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:15:17.135824 23776 raft_consensus.cc:2889] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:17.147289 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66467.3us
I20260709 12:15:17.128655 28870 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.154117 29013 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 12:15:17.159286 29058 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.862s	user 0.002s	sys 0.000s
I20260709 12:15:17.132274 28576 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:17.160090 28576 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.162781 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 505) took 1566 ms. Trace:
I20260709 12:15:17.163216 25170 rpcz_store.cc:276] 0709 12:15:15.596784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.597834 (+  1050us) service_pool.cc:224] Handling call
0709 12:15:15.625890 (+ 28056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.626274 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.626304 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.633371 (+  7067us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.036073 (+402702us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:16.973457 (+937384us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b1800396d60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2ad35a4bc 00007fb2aa6a848b 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2b11aa3c7 00007fb2ad7bc9a7 00007fb2ad7cc0bf
0709 12:15:16.973603 (+   146us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.973840 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.973917 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:17.158488 (+184571us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:17.162543 (+  4055us) raft_consensus.cc:1685] finished
0709 12:15:17.162567 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:17.162711 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411,"prepare.queue_time_us":192193,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":5510,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":43018,"spinlock_wait_cycles":13717120,"thread_start_us":535958,"threads_started":2,"wal-append.queue_time_us":581039}
I20260709 12:15:17.164342 28886 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.165048 29070 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.166502 29109 tablet_replica.cc:333] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:15:17.167629 29109 raft_consensus.cc:2243] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:17.177309 28831 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.177610 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:17.146394 29067 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:15:17.180485 28984 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.131544 29066 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:17.148391 29088 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:15:17.183197 28926 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.183512 29067 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:17.184269 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.186543 28860 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.188526 28318 consensus_queue.cc:1048] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:17.189502 28621 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:17.191241 28815 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.182462 28947 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:15:17.196827 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.212011 23777 raft_consensus.cc:2410] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:15:17.198989 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.208060 23751 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.210901 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.215523 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.216570 23785 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:17.195133 28946 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.219085 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.219938 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.224692 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.226012 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.226882 23782 raft_consensus.cc:2410] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:15:17.229903 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.230786 23777 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 9: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:17.205950 29053 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:17.235394 29053 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.238094 29053 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.239821 29082 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:17.221158 29067 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:17.240851 28886 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.211694 28621 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:17.241586 29066 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.241874 28621 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.242697 29066 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:17.242874 28621 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:17.213631 28926 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.245512 29067 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.235790 29051 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:17.148159 29064 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.249107 23782 raft_consensus.cc:1217] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.8->[11.9-14.11]   Dedup: 11.9->[12.10-14.11]
I20260709 12:15:17.251128 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.253382 23782 pending_rounds.cc:85] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 9
I20260709 12:15:17.254029 23782 pending_rounds.cc:107] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:15:17.254606 23782 raft_consensus.cc:2889] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:17.214792 28815 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.257265 28737 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.257323 28815 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.267669 28984 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.215811 28860 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.270952 28860 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:17.285389 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 189065us
I20260709 12:15:17.204423 28987 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.228749 29100 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:17.298241 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.298640 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.298900 23927 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:17.299265 23931 raft_consensus.cc:1275] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:17.299623 23932 raft_consensus.cc:2410] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:17.300495 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.301249 23934 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
W20260709 12:15:17.305239 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 271188us
I20260709 12:15:17.246656 28926 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:17.361405 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 87943.1us
I20260709 12:15:17.364432 23779 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:15:17.364715 23933 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:15:17.366667 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:17.368837 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.369943 28690 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:17.370718 28690 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.372131 29052 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:17.372785 29052 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.373878 28690 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:17.384439 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:17.385422 25113 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.250658 28756 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.385028 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:17.393369 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.396662 28946 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.397347 29053 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:17.397727 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 49448.1us
I20260709 12:15:17.398082 29088 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:17.397362 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.397511 28946 consensus_queue.cc:1048] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:17.398869 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.400619 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.406032 23898 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.365211 28576 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:17.410638 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 385265us
I20260709 12:15:17.413316 23857 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:15:17.406651 23784 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:17.415552 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.416738 23857 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:15:17.418504 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.397884 28882 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:17.420086 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.421020 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.421141 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.398535 29088 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.389149 29021 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:17.423223 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.424327 29086 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:17.377137 28968 consensus_queue.cc:1059] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:15:17.429973 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.407375 29070 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.432567 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.461545 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.351966 29100 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.407121 28370 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:15:17.418133 28690 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:17.409535 28886 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:17.495718 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.459182 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.482111 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:17.409337 29053 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.430987 29088 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.502722 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:17.506410 25174 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:17.433939 28756 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.423476 28882 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:17.502617 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65740.5us
I20260709 12:15:17.503021 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:17.504720 25119 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.505497 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.430753 29109 raft_consensus.cc:2272] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:17.514753 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.515210 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.518823 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.458779 28851 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:17.515903 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.470942 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.522203 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.515753 23928 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:17.519388 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.526554 23927 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:17.536307 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.537038 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.434192 29051 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:17.467868 29004 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:17.538371 23853 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:17.496338 29013 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:17.540254 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.541424 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.564530 23777 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:17.560793 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.565769 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.547956 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.551646 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.552934 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.555992 29064 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.556516 23857 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:17.569440 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.556746 28987 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.574856 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.557705 23927 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:17.557893 29034 raft_consensus.cc:493] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:17.558221 29100 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.558929 23899 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.559482 28884 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:15:17.588657 29118 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:15:17.588836 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:17.590533 29125 log.cc:927] Time spent T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:15:17.590942 29109 ts_tablet_manager.cc:1916] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:17.509366 28984 consensus_queue.cc:1059] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
W20260709 12:15:17.559886 29119 log.cc:927] Time spent T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:15:17.594383 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:17.594554 29034 raft_consensus.cc:515] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.472726 29070 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.596820 29034 leader_election.cc:290] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.597162 29103 raft_consensus.cc:3055] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:17.597630 29070 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.597970 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.598624 23780 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 5.
I20260709 12:15:17.566035 29055 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:17.536803 28886 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.601210 23931 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:17.601730 23860 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:17.601894 28370 consensus_queue.cc:1059] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:17.602684 28886 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.604220 28860 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:17.604645 28860 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:17.606805 28984 raft_consensus.cc:3055] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:17.607244 28984 raft_consensus.cc:740] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:17.557629 28690 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:17.597556 29103 raft_consensus.cc:740] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:17.608064 28690 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:17.610659 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 53740.9us
I20260709 12:15:17.615574 25174 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:17.618409 23783 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:17.520319 23933 raft_consensus.cc:2376] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 9. Re-sending same reply.
I20260709 12:15:17.566915 23852 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:17.562456 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.575994 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:17.620483 23858 raft_consensus.cc:2393] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:17.621236 23853 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:17.576419 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:17.469440 28946 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:15:17.579478 25170 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:17.433996 29086 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.518466 28882 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.578792 23857 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:15:17.465287 29021 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.583498 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.584858 29051 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:17.576716 29112 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:15:17.622905 23927 raft_consensus.cc:606] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.564157 28576 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:17.559772 28870 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:17.563766 28949 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:17.573874 29115 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.654616 23929 raft_consensus.cc:1217] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Deduplicated request from leader. Original: 13.7->[15.8-15.8]   Dedup: 15.8->[]
I20260709 12:15:17.655169 23929 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:15:17.655861 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:17.630352 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71503.3us
W20260709 12:15:17.629577 29116 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.474s	user 0.004s	sys 0.000s
I20260709 12:15:17.579957 28741 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.578684 29013 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.661809 28741 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.631239 28737 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.537254 29098 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.211s
I20260709 12:15:17.664083 28741 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.664924 29013 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.656878 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:17.651975 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.635792 28999 consensus_queue.cc:1059] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:15:17.645838 28550 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.668829 29013 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:17.635246 29122 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.328s	user 0.005s	sys 0.000s
I20260709 12:15:17.669463 29013 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:17.653524 28949 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:17.670969 28949 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:17.673202 28935 consensus_queue.cc:1059] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:17.634490 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:17.628715 29086 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.645184 29021 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:17.638567 29123 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 12:15:17.681190 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46758.5us
I20260709 12:15:17.692013 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.692833 23932 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:17.674875 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.712502 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.713869 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.718408 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.719887 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.643517 29053 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.767094 23777 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 8. Current term is 9. Ops: []
W20260709 12:15:17.762827 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 80496.9us
I20260709 12:15:17.760571 28741 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:17.770269 28741 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:17.771229 28741 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.775128 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.651868 28690 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:17.776484 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.769620 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:17.777802 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.609503 29103 consensus_queue.cc:260] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.778268 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.710378 28550 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:17.780742 23825 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:17.783843 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.780234 25176 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:17.796725 25176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.654287 29115 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.785611 23928 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:17.785755 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:17.785760 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 110976us
I20260709 12:15:17.808969 25174 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:17.788815 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.789129 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:17.792245 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:17.793049 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.785444 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.673062 28851 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:15:17.784857 23931 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:17.811437 25172 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:17.813410 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.800972 23751 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.813732 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.605509 28860 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:17.653074 28870 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:17.814517 23931 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:17.816411 23780 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:17.827553 28884 raft_consensus.cc:993] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:17.815940 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:17.820158 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:17.779762 28926 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.841415 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:17.842728 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 338091us
W20260709 12:15:17.843988 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 51050us
I20260709 12:15:17.851379 23925 raft_consensus.cc:2393] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:17.851554 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:17.844909 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44023.1us
I20260709 12:15:17.853662 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.854617 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:17.651918 28984 consensus_queue.cc:260] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:17.804631 29065 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.868201 23860 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:17.796195 28935 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:17.870153 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.871488 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:17.818272 29051 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:17.891202 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:17.838323 29013 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:17.896173 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.862336 29053 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:17.899917 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 79839.2us
W20260709 12:15:17.902688 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 122037us
I20260709 12:15:17.880131 23778 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:17.941888 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:17.940382 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:17.915498 25174 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 10.
W20260709 12:15:17.930169 29122 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:15:17.865657 28860 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:18.000125 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59857us
I20260709 12:15:18.000974 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:17.900547 28823 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:17.893221 29136 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:17.954106 29092 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:15:17.895922 28831 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:18.012686 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 121578us
I20260709 12:15:18.014918 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:17.942469 28947 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:17.906117 29051 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:18.007155 29123 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:15:18.006840 29087 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.951476 28956 consensus_queue.cc:1059] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
W20260709 12:15:18.021660 28621 consensus_meta.cc:287] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.778s	user 0.013s	sys 0.000s
I20260709 12:15:18.013636 29136 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:18.018453 28766 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.000551 28741 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:15:17.999104 28974 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:15:18.025710 29013 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:18.018710 28980 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:18.025370 28823 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:18.065869 29135 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:18.067111 29126 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 12:15:18.192612 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:18.126173 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.095116 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:18.123765 28756 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.001s
W20260709 12:15:18.205999 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 110942us
I20260709 12:15:18.074786 29053 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:18.188393 29066 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.930s	user 0.011s	sys 0.001s
I20260709 12:15:18.231755 23781 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:18.058888 28870 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:18.213052 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.222648 28621 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:18.099485 29136 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:18.065771 29134 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:15:18.190810 29139 log.cc:927] Time spent T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:15:18.092468 29137 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.193617 28823 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.228876 28741 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:18.214684 29053 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:18.100500 28766 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.107185 28882 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.108325 28949 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.238960 28621 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:15:18.263609 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 262697us
I20260709 12:15:18.269718 28335 raft_consensus.cc:3055] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:18.270212 28335 raft_consensus.cc:740] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:18.259183 28741 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.274152 28741 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.278184 28882 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.278995 25171 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:18.279899 25176 raft_consensus.cc:1240] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:15:18.281144 28823 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.284870 28766 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.287611 28949 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:18.288971 23824 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.291256 28823 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:18.260551 29134 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:18.291896 28823 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:18.292202 28980 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:18.293071 28823 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.293862 28980 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:18.294379 28980 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:18.296394 29088 consensus_meta.cc:287] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.001s
I20260709 12:15:18.260908 29086 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.296962 29088 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.297906 23934 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:18.298637 25173 raft_consensus.cc:1240] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:18.267310 28621 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:18.300083 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.260051 29053 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:18.271100 28335 consensus_queue.cc:260] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.051028 29051 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:18.235307 23777 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:18.192256 29013 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:18.029018 28947 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.062090 28956 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:18.059549 29052 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.072510 29115 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.242266 25175 raft_consensus.cc:2410] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:15:18.028520 28831 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:18.247294 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 371093us
I20260709 12:15:18.292243 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.302582 28335 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:18.267445 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.324116 29088 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.307607 28766 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.329329 29013 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:18.365370 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 97997.5us
I20260709 12:15:18.368214 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.328121 29086 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.331883 28956 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:18.377521 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 362597us
I20260709 12:15:18.377996 23825 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:18.370061 28974 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:18.388528 29161 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.389196 29161 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.352475 29135 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.393384 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:18.400035 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:18.391829 29087 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:18.410116 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:18.392277 28998 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:18.371088 28766 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.468420 25172 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:18.358144 28980 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.493404 28980 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:18.494445 23776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 1021) took 1126 ms. Trace:
I20260709 12:15:18.494874 23776 rpcz_store.cc:276] 0709 12:15:17.368295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.652680 (+284385us) service_pool.cc:224] Handling call
0709 12:15:17.653131 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.654442 (+  1311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.654482 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.658026 (+  3544us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.067801 (+409775us) spinlock_profiling.cc:243] Waited 899 us on lock 0x7b180047b3c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:18.332948 (+265147us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:18.332996 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.334120 (+  1124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.494039 (+159919us) raft_consensus.cc:1685] finished
0709 12:15:18.494061 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.494375 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3132,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":2188,"spinlock_wait_cycles":2066688,"thread_start_us":264909,"threads_started":1,"wal-append.queue_time_us":743565}
I20260709 12:15:18.496461 29137 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.497176 29137 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.457297 29161 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.484311 28766 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:18.487718 29157 log.cc:927] Time spent T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:15:18.446435 29152 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:18.464450 29066 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.402112 29135 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:18.526481 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 56596.8us
I20260709 12:15:18.457098 28956 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.536190 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.521317 29109 ts_tablet_manager.cc:1929] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 8.6
I20260709 12:15:18.548712 28831 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.551862 29109 log.cc:1199] T 4e60a1c6520545729d27474d8f237ce5 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/4e60a1c6520545729d27474d8f237ce5
I20260709 12:15:18.554411 29109 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50860 (request call id 0) took 1590 ms. Trace:
I20260709 12:15:18.529803 29066 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.501081 29156 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:15:18.547549 28947 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.566592 29053 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.570931 29087 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:18.581732 29170 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:18.463629 29134 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.583889 29137 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.584259 29135 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.585713 29136 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.588619 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2951) took 1193 ms. Trace:
I20260709 12:15:18.664796 23783 rpcz_store.cc:276] 0709 12:15:17.395062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.764172 (+369110us) service_pool.cc:224] Handling call
0709 12:15:17.764514 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.768281 (+  3767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.768317 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.768790 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.318441 (+549651us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b1800488860. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad3520d1 00007fb2ad76da0f
0709 12:15:18.419416 (+100975us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:18.468968 (+ 49552us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.469174 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.469248 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.588260 (+119012us) raft_consensus.cc:1685] finished
0709 12:15:18.588398 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.588551 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19023,"prepare.queue_time_us":49736,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":57106,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":23390,"spinlock_wait_cycles":12657920,"thread_start_us":100736,"threads_started":1,"wal-append.queue_time_us":787442}
I20260709 12:15:18.582651 28621 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:18.668391 29159 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:15:18.670542 28621 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.614411 28947 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.671561 25177 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 711) took 1191 ms. Trace:
I20260709 12:15:18.671787 28947 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.674454 28947 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.602097 28886 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.676080 28886 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.611217 29051 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:18.622993 29160 log.cc:927] Time spent T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:15:18.628340 23776 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:18.628875 23781 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:18.630381 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:18.631951 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:18.680689 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:18.681208 23785 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:18.634315 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:18.554925 29109 rpcz_store.cc:276] 0709 12:15:16.964422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.965036 (+   614us) service_pool.cc:224] Handling call
0709 12:15:17.149978 (+184942us) ts_tablet_manager.cc:1151] Deleting tablet 4e60a1c6520545729d27474d8f237ce5
0709 12:15:17.166045 (+ 16067us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:15:17.772812 (+606767us) tablet_metadata.cc:665] Metadata flushed
0709 12:15:18.512072 (+739260us) tablet_metadata.cc:665] Metadata flushed
0709 12:15:18.554297 (+ 42225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179916,"tablet-delete.queue_time_us":129487,"thread_start_us":50463,"threads_started":1}
I20260709 12:15:18.637221 28974 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.688414 28974 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.690626 28974 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.648455 29066 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.649557 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.650712 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:18.653275 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.691675 23782 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 5.
I20260709 12:15:18.654094 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.670802 28810 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:18.693130 28810 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:18.694164 29023 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:18.694559 28886 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.698762 29136 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:18.702411 23810 heartbeater.cc:646] Failed to heartbeat to 127.22.25.126:39113 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.22.25.126:39113 timed out after 15.000s (SENT)
I20260709 12:15:18.677505 23784 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:18.676704 29087 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.706336 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.629591 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:18.680034 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:18.707573 23776 raft_consensus.cc:2410] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:15:18.667227 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:18.708037 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:18.702373 29051 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.709245 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:18.709682 29051 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.710350 29051 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.707273 23779 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:18.677505 25177 rpcz_store.cc:276] 0709 12:15:17.480381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.481709 (+  1328us) service_pool.cc:224] Handling call
0709 12:15:17.495123 (+ 13414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.678815 (+183692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.678852 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.679442 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.246909 (+567467us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:18.246953 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.247029 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.667775 (+420746us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:18.671230 (+  3455us) raft_consensus.cc:1685] finished
0709 12:15:18.671254 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.671493 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27227,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":24646,"spinlock_wait_cycles":6508416,"thread_start_us":545853,"threads_started":2,"wal-append.queue_time_us":548826}
I20260709 12:15:18.713490 29053 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.713541 29051 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.717022 29013 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.726995 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:18.728394 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:18.729425 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:18.690667 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.742868 23851 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:18.743451 23857 raft_consensus.cc:2410] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:15:18.741348 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:18.744971 29023 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.746750 29023 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.747495 29120 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.749467 29023 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.745626 29066 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.751500 29066 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.757061 29066 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.757891 29066 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:18.758893 29066 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.761672 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:18.764464 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.745505 28886 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:18.817582 29170 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:18.789220 29023 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:18.818894 29023 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:18.820802 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.821498 28550 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:18.821888 28550 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:18.822249 28550 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:18.824590 23784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40134 (request call id 1027) took 1157 ms. Trace:
I20260709 12:15:18.824972 23784 rpcz_store.cc:276] 0709 12:15:17.667332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.113997 (+446665us) service_pool.cc:224] Handling call
0709 12:15:18.195095 (+ 81098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.703983 (+508888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.824349 (+120366us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540010c554. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:18.824526 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108980480}
I20260709 12:15:18.826027 29095 raft_consensus.cc:1081] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:18.789508 28974 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:18.828127 28974 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:18.828738 28974 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:18.831382 28886 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:15:18.833829 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.834358 29066 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.838374 28984 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:18.840442 28968 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.841192 28968 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.841861 28851 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.842540 28851 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.819330 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 3030) took 1035 ms. Trace:
I20260709 12:15:18.848747 23932 rpcz_store.cc:276] 0709 12:15:17.783762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.787175 (+  3413us) service_pool.cc:224] Handling call
0709 12:15:17.787446 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.787675 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:17.846592 (+ 58917us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.818814 (+972222us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b180042baa0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:15:18.818962 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:18.819014 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:18.819062 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.819128 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.819267 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47856,"prepare.queue_time_us":58671,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":8257,"raft.queue_time_us":979266,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":5506816}
I20260709 12:15:18.852432 23930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55680 (request call id 722) took 1555 ms. Trace:
I20260709 12:15:18.852958 23930 rpcz_store.cc:276] 0709 12:15:17.296599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.297933 (+  1334us) service_pool.cc:224] Handling call
0709 12:15:17.298274 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.298487 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:17.358916 (+ 60429us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.039751 (+680835us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.852024 (+812273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:18.852098 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:18.852153 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.852226 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.852361 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1823197,"prepare.queue_time_us":153032,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":13783,"raft.queue_time_us":562,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":21432,"spinlock_wait_cycles":42752,"thread_start_us":628427,"threads_started":2,"wal-append.queue_time_us":648791}
I20260709 12:15:18.845969 28851 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:18.790508 29051 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:18.856310 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 48010us
I20260709 12:15:18.863826 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:18.864182 23933 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 3033) took 1079 ms. Trace:
I20260709 12:15:18.864627 23926 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:18.865242 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.866480 23927 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:18.867797 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.868652 23930 raft_consensus.cc:2393] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:18.870306 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:18.870913 23927 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:18.872124 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.864670 23933 rpcz_store.cc:276] 0709 12:15:17.784608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.785014 (+   406us) service_pool.cc:224] Handling call
0709 12:15:17.834160 (+ 49146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.834388 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:18.240294 (+405906us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.863780 (+623486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:18.863848 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:18.863899 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.863974 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.864115 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":465011,"prepare.queue_time_us":165369,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"thread_start_us":530701,"threads_started":2,"wal-append.queue_time_us":259601}
I20260709 12:15:18.884076 29023 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:18.808490 29013 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.885552 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.886313 23932 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:18.892729 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:18.892757 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.886387 28335 raft_consensus.cc:493] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.897511 28335 raft_consensus.cc:515] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:18.894201 23927 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:18.902660 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:18.904276 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 231293us
I20260709 12:15:18.907941 23926 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:18.913650 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.916005 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:18.877323 29051 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:18.922868 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.923619 23926 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:18.926422 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:18.910815 28335 leader_election.cc:290] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:18.972807 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:18.931831 23825 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:18.913882 28968 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:18.975282 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.976632 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:18.974246 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:18.980934 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:18.978258 29098 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:18.929574 28971 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:18.982532 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50839.3us
I20260709 12:15:18.982458 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:18.992602 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:18.993265 25177 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:18.991508 23749 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:18.996498 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:18.917392 28621 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:15:18.978288 29170 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:19.000049 23932 raft_consensus.cc:1240] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:18.919072 29153 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:15:18.998840 23926 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:18.921455 28766 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:15:18.975173 28851 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:18.999001 23749 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:18.988188 29067 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.007640 25119 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.008317 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:19.016850 29176 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:15:19.018324 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.019200 23925 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:18.994040 28971 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:19.020123 28971 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.022502 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.023200 23928 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:19.008581 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.025154 23783 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:18.971810 29100 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:15:18.990643 29174 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:19.009325 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:18.998077 29016 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:19.002713 29170 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.000504 29082 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:19.002290 29135 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:19.030747 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:19.032683 23926 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:19.032305 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.033983 28944 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.024276 29023 raft_consensus.cc:2749] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:19.009987 29067 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:18.995189 29098 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.054535 28766 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:19.037925 29170 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:19.062866 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.032259 29174 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:19.067574 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 713) took 1586 ms. Trace:
I20260709 12:15:19.046078 29135 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.068856 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 3017) took 1557 ms. Trace:
I20260709 12:15:19.082325 23934 rpcz_store.cc:276] 0709 12:15:17.511346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.513387 (+  2041us) service_pool.cc:224] Handling call
0709 12:15:17.513721 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.298294 (+784573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.068584 (+770290us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54000e3154. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:19.068793 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802766720}
I20260709 12:15:19.024101 28851 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.081463 25173 rpcz_store.cc:276] 0709 12:15:17.480980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.482298 (+  1318us) service_pool.cc:224] Handling call
0709 12:15:17.482615 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.299039 (+816424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.067272 (+768233us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54006d0e54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:19.067501 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876107136}
I20260709 12:15:19.074803 25177 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:19.068305 23780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2959) took 1665 ms. Trace:
I20260709 12:15:19.053444 29082 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:19.051573 29016 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:19.085130 23780 rpcz_store.cc:276] 0709 12:15:17.402563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.840637 (+438074us) service_pool.cc:224] Handling call
0709 12:15:17.841091 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.006993 (+165902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.007025 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.007500 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.266883 (+259383us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:18.266928 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.067893 (+800965us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54000bd454. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:19.067946 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:19.067995 (+    49us) raft_consensus.cc:1685] finished
0709 12:15:19.068008 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.068237 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106797,"raft.queue_time_us":113896,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9617,"spinlock_wait_cycles":122113920,"thread_start_us":257377,"threads_started":1,"wal-append.queue_time_us":260149}
I20260709 12:15:19.058156 29067 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.084915 29174 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.087477 29135 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.084374 29184 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:19.124776 29179 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:15:19.127034 28956 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.127494 28824 raft_consensus.cc:3055] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:19.127930 28824 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:19.128715 28824 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.131605 29082 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.101356 29085 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:19.136466 29085 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.138713 29085 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.140159 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:19.141624 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:19.142804 23851 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:19.143373 23851 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:19.143746 23851 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.148423 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.150568 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:19.151191 23934 raft_consensus.cc:606] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:19.151533 23934 raft_consensus.cc:614] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:19.153255 23852 raft_consensus.cc:1240] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:19.149106 23853 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:19.187762 23859 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:19.212004 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.206873 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.208736 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.163023 28824 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.143757 28851 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.219218 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.227651 23900 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.227535 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.236109 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:19.236486 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 2954) took 1835 ms. Trace:
I20260709 12:15:19.237010 23777 rpcz_store.cc:276] 0709 12:15:17.401345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.784952 (+383607us) service_pool.cc:224] Handling call
0709 12:15:17.785249 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.306833 (+521584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.236233 (+929400us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54001eebd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:19.236421 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034171648}
I20260709 12:15:19.209273 28831 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.202070 29051 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.179543 23851 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.123082 28810 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.205747 28947 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.250584 23857 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:19.256181 23851 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.238684 28882 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.239490 28851 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.234684 28974 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.243439 29085 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:19.147361 29184 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.265163 28882 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.273789 23856 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:15:19.210662 29016 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.257956 29051 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.244206 29186 consensus_queue.cc:1059] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:15:19.260946 28947 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.255990 28949 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:15:19.283097 28882 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:19.284816 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 1.34377e+06us
I20260709 12:15:19.291366 28370 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:19.315352 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.316108 23929 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
W20260709 12:15:19.316653 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.316771 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:19.317317 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.295814 28318 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.296554 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.275172 29085 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.309616 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:19.318208 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.318398 25172 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:19.330435 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.308497 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:19.321498 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.317658 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.322688 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:19.323210 23929 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:19.334218 25174 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.254405 28831 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.324134 25175 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:19.336010 23928 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:15:19.336447 23928 pending_rounds.cc:85] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 9
I20260709 12:15:19.336663 23928 pending_rounds.cc:107] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:15:19.336889 23928 raft_consensus.cc:2889] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:19.337244 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.338042 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:19.338632 23928 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:19.338909 23928 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:19.345007 28884 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:19.346601 29187 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:15:19.327111 29085 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.355480 29103 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.360278 28882 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:19.360929 28882 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.362088 28810 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.363240 28882 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.302635 28974 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:19.369340 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60953.5us
I20260709 12:15:19.371791 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.366995 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.378260 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:19.311110 28968 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.384392 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.385260 23855 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 15: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:19.312302 28370 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:19.394594 25174 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:19.374122 23928 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.401206 23928 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.326869 29098 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.347712 28947 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.327631 28318 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.310421 29051 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.406530 29184 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.381085 28884 raft_consensus.cc:2764] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:15:19.475884 28926 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.478502 28926 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.473937 28851 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:19.480356 28318 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.480531 28851 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:19.446434 28370 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.482757 28370 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:19.448421 28984 consensus_meta.cc:287] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
I20260709 12:15:19.486061 23860 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:19.489320 28737 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:19.450906 28550 consensus_meta.cc:287] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.004s
I20260709 12:15:19.492933 28550 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.495183 28851 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.497588 28851 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.499552 29051 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.499827 23933 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:19.500293 29051 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.476106 28815 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.500592 23934 raft_consensus.cc:606] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.500870 29051 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.501472 23934 raft_consensus.cc:614] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:19.501480 28947 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.502671 28947 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:19.504396 23860 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56648 (request call id 1580) took 1634 ms (client timeout 2094 ms). Trace:
I20260709 12:15:19.504806 28926 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.048583 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 4e60a1c6520545729d27474d8f237ce5 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:19.498193 28550 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:19.542344 23668 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 506420us
I20260709 12:15:19.544593 29184 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:19.545395 29184 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.545449 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:19.549782 25177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:19.549556 28884 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:15:19.552345 29086 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:19.552886 29086 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:19.554167 29086 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.557065 29086 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:19.506763 28947 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.508255 28815 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.509987 28831 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.463680 23931 raft_consensus.cc:1275] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:19.491940 28737 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:19.570045 28737 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:19.514357 28935 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:19.516924 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.571013 28935 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:19.517469 23926 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:15:19.518303 29189 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.170s	user 0.003s	sys 0.000s
I20260709 12:15:19.572343 28935 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.574074 28935 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.575532 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.576244 23780 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:19.518994 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.519794 29066 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.580044 29066 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.584143 29066 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.522943 29056 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.523726 29053 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.585530 29056 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.585963 29053 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.589211 29053 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.589741 29056 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.447346 28335 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.533320 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.536010 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:19.523867 29188 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.205s	user 0.002s	sys 0.000s
I20260709 12:15:19.590790 28335 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.593228 23784 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
W20260709 12:15:19.504829 23860 rpcz_store.cc:269] 0709 12:15:17.869510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.871244 (+  1734us) service_pool.cc:224] Handling call
0709 12:15:19.504153 (+1632909us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54000cc1d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:19.504289 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3711276416}
I20260709 12:15:19.595412 28335 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.512425 25172 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:19.596015 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.597066 23780 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:19.597822 25177 raft_consensus.cc:606] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:19.598269 25177 raft_consensus.cc:614] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:19.600840 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:15:19.601414 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:19.604401 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 65868.1us
I20260709 12:15:19.605121 29135 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.595767 23683 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 11, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.606850 23779 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:19.609195 28815 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.595507 29053 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.610503 29053 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.612838 29184 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.441561 28971 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.615909 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:19.621897 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.622545 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.623148 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.622646 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.625034 23860 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:19.629716 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.630473 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.630445 25172 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:19.632505 23928 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:19.632969 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.637395 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.639072 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:15:19.637627 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 246212us
I20260709 12:15:19.639961 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:19.647946 23683 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 14, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:19.649497 23683 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 9, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 9 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:19.655125 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.656807 23930 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:15:19.657917 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:19.662256 23932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.074s	sys 0.107s
W20260709 12:15:19.662698 23932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.074s	sys 0.107s
I20260709 12:15:19.503314 29051 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.669252 25174 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:19.671135 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:19.651017 23683 catalog_manager.cc:5697] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:19.673043 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 49296.3us
I20260709 12:15:19.672941 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:19.675441 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:19.676126 23683 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:19.678480 23683 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 13, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:19.679124 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:19.680639 23683 catalog_manager.cc:5697] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 14, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.614154 29184 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.610388 29135 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.684517 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:19.690209 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.685751 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 44508.5us
I20260709 12:15:19.691191 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:19.682529 23683 catalog_manager.cc:5697] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.407229 28974 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.694259 23683 catalog_manager.cc:5697] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.697201 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:19.697698 28974 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.698406 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:19.455984 29170 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.692108 28926 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:15:19.697685 28882 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.700785 28882 raft_consensus.cc:493] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.701177 28882 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.698457 23683 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 14, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 14 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:19.698753 29181 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.703629 23683 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 15, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.683411 28815 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.686429 29088 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:19.676369 28756 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.705358 23683 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 12, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.389369 28968 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.706431 28756 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.707195 23929 raft_consensus.cc:1240] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:19.695070 28831 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.708201 28831 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.708776 28831 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.709426 23683 catalog_manager.cc:5697] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.692124 28737 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:15:19.663429 28810 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.704303 29181 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:19.714422 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.729585 23856 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:19.672550 28971 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:19.732157 28824 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:19.733546 29022 consensus_queue.cc:1059] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:19.723898 28980 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:19.734442 28971 raft_consensus.cc:3055] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:19.734905 28971 raft_consensus.cc:740] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:19.403029 29186 consensus_queue.cc:1175] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "438159d8de3c4027bdcf57eb81dedf86" responder_term: 11 status { last_received { term: 10 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:19.686663 28884 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:19.686859 29133 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:19.736217 29133 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.735916 28971 consensus_queue.cc:260] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.736631 29133 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:19.737452 28971 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.734867 29181 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.712987 29135 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:19.720606 23683 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.709151 28968 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:19.687493 29184 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:19.691248 29051 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.693655 29053 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.585187 29174 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.728451 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:19.729996 28810 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.741183 28884 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.742314 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:19.704978 28815 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.711803 28974 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.727797 28756 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.734530 28980 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:19.769163 23751 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.66:44897 timed out after 2.094s (SENT) [suppressed 10 similar messages]
I20260709 12:15:19.747619 23683 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.770673 23683 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:19.723937 28984 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:19.776281 23683 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:19.779184 23681 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 274) took 42548 ms (client timeout 15000 ms). Trace:
W20260709 12:15:19.786204 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.785646 23683 catalog_manager.cc:5697] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.760660 29184 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.787858 23683 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:19.764081 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:19.768504 28999 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:19.770438 29053 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:19.773543 28335 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:19.773892 29170 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.775238 28980 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.768357 28884 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:19.798326 28621 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:19.784018 28984 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:19.794037 28999 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:19.781464 23681 rpcz_store.cc:269] 0709 12:14:37.230995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.231133 (+   138us) service_pool.cc:224] Handling call
0709 12:14:37.998951 (+767818us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:19.779097 (+41780146us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:16.796489 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:16.797079 (+   590us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:16.797113 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:15:16.797125 (+    12us) tablet.cc:662] Decoding operations
0709 12:15:16.798143 (+  1018us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:16.798210 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:16.798241 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:15:16.799058 (+   817us) tablet.cc:701] Row locks acquired
0709 12:15:16.799077 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:15:16.799286 (+   209us) write_op.cc:270] Start()
0709 12:15:16.799443 (+   157us) write_op.cc:276] Timestamp: P: 1783599316799206 usec, L: 0
0709 12:15:16.799463 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:15:16.854319 (+ 54856us) log.cc:844] Serialized 4000 byte log entry
0709 12:15:17.003214 (+148895us) op_driver.cc:464] REPLICATION: finished
0709 12:15:17.344039 (+340825us) write_op.cc:301] APPLY: starting
0709 12:15:17.377359 (+ 33320us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:17.582829 (+205470us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:17.582926 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:17.770012 (+187086us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:17.770256 (+   244us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:15:17.770301 (+    45us) write_op.cc:312] APPLY: finished
0709 12:15:17.770577 (+   276us) log.cc:844] Serialized 97 byte log entry
0709 12:15:17.770790 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:17.770942 (+   152us) write_op.cc:454] Released schema lock
0709 12:15:17.770996 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:17.771436 (+   440us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:17.771465 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"spinlock_wait_cycles":974402816,"child_traces":[["op",{"apply.queue_time_us":285657,"apply.run_cpu_time_us":18796,"apply.run_wall_time_us":468454,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":740704,"prepare.run_cpu_time_us":26876,"prepare.run_wall_time_us":157268,"raft.queue_time_us":28932,"raft.run_cpu_time_us":72665,"raft.run_wall_time_us":200109,"replication_time_us":203551,"spinlock_wait_cycles":1985408,"thread_start_us":620670,"threads_started":4,"wal-append.queue_time_us":99754,"wal-append.run_cpu_time_us":9170,"wal-append.run_wall_time_us":1826231}]]}
I20260709 12:15:19.761998 29135 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.809178 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:19.770227 29051 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.818920 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:19.819777 28886 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.820513 28886 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:19.822908 28737 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:19.823510 28737 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:19.824532 28737 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.800367 28335 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.831194 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.832204 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.832700 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.833310 23851 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
W20260709 12:15:19.811375 29082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.072s	sys 0.121s
I20260709 12:15:19.832845 23855 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:19.806560 28971 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:15:19.810209 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e60a1c6520545729d27474d8f237ce5 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:50860
I20260709 12:15:19.836516 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.835935 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:19.836406 23860 raft_consensus.cc:1240] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:19.838342 23851 raft_consensus.cc:2393] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:19.811993 28984 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:19.813352 28999 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:19.842757 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:19.842911 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 421915us
I20260709 12:15:19.844542 23860 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:19.842756 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.845434 23851 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:19.846662 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:19.848204 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 171371us
I20260709 12:15:19.806803 28956 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:15:19.802603 29053 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.855798 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.859066 23855 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:15:19.861835 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.864588 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.864944 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.865483 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:19.865828 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:19.865521 23778 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:19.870841 28882 raft_consensus.cc:515] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.834863 28335 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.876281 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.877066 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:19.877645 23783 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:19.855054 23860 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:19.880896 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.882378 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:19.882452 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.883889 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:19.835192 29082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.072s	sys 0.121s
I20260709 12:15:19.879037 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.885021 23860 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:19.884717 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.887089 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:15:19.873903 29067 consensus_meta.cc:287] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.003s
I20260709 12:15:19.883224 23931 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.874925 29016 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:15:19.896190 29191 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:19.886731 28756 consensus_queue.cc:1059] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:15:19.897850 28756 raft_consensus.cc:3055] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:19.898200 28756 raft_consensus.cc:740] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:19.901070 23781 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:19.901400 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.873349 28810 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:19.876305 29170 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:19.904903 23931 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:19.908228 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:19.908437 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 192949us
I20260709 12:15:19.904295 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.908123 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.910234 23855 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:19.911209 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:19.853804 28886 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:15:19.913812 23777 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:19.915393 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:19.906638 29085 consensus_meta.cc:287] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.006s
I20260709 12:15:19.922955 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:19.924975 25174 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:19.899048 28756 consensus_queue.cc:260] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.927659 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f5f207bbfed4f119db8464be0a5a8cf" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:19.922484 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 153329us
W20260709 12:15:19.929270 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.931105 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.935040 23783 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:19.935634 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:19.935829 23782 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:19.907785 29170 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.944810 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:19.948686 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:19.949757 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.949525 23778 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:19.951872 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.953589 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872245a7005f472bb7f6cfafda680b74" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.955456 23783 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:19.959715 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.960543 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.961352 23783 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:19.974882 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.975731 23778 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:19.976696 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.980863 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:19.982209 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:19.983691 23778 raft_consensus.cc:2393] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:19.902321 29067 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:19.987044 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55575.4us
I20260709 12:15:19.989288 23928 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:20.035686 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.993388 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:19.993602 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:19.994825 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:19.943853 29170 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:19.995749 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:20.037417 23852 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.913991 29181 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:19.945025 29174 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.039410 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.040527 25174 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:20.041821 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:19.928586 29085 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:19.984977 28756 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.064793 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:20.049266 23930 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:20.050369 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:20.065522 23783 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:20.065964 23928 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:20.067037 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.069005 23779 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:15:20.067037 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:20.071244 23852 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:20.072150 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 163040us
I20260709 12:15:20.071835 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:20.044062 29095 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:20.031016 29016 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.998996 29194 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.031087 28815 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.045765 29066 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:20.044237 28949 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.100260 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.062418 29174 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.079206 28318 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:20.107751 28851 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.004s
W20260709 12:15:20.138474 29192 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:15:20.114591 29066 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:20.105379 28737 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:15:20.090975 29095 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:20.141536 29103 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.000s
I20260709 12:15:20.106549 28949 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.140363 28882 leader_election.cc:290] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.149287 29085 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.118330 29174 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.172078 29185 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.103104 29194 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:20.188100 29195 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:15:20.187770 29133 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.190080 28974 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:20.190668 28974 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:20.191514 28949 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.190169 29133 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.195087 29066 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.196237 29181 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.185974 29185 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.208446 29184 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.198132 29194 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.198489 29095 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.210974 29184 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:20.262135 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 185413us
I20260709 12:15:20.265651 23825 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:20.268316 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:20.270282 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:20.262821 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:20.275566 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.279270 23927 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:20.279723 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:20.282223 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:20.282835 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.265508 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:20.284700 23784 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.286710 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 4e60a1c6520545729d27474d8f237ce5 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:20.288486 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:20.290633 23855 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.293084 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.294857 23933 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:20.296285 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.296365 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.297983 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.299081 23933 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:20.309582 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:20.309785 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.312122 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:20.313019 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.313433 25177 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16.
I20260709 12:15:20.316270 23898 leader_election.cc:400] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16.
I20260709 12:15:20.316751 23898 leader_election.cc:403] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:15:20.318063 25177 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:20.319952 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:20.321125 25177 raft_consensus.cc:2393] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:20.323487 25113 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:20.323524 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.326344 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.333540 23933 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:20.353780 25113 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:20.361673 28815 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.362928 28815 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.373178 29095 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.373915 29095 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.378669 29103 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:20.386626 23784 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:20.390265 29209 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:20.391278 29209 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:20.391484 29198 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:20.393821 29209 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.395758 29095 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.405942 29088 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:20.369498 29218 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.154s	user 0.003s	sys 0.000s
I20260709 12:15:20.422632 23855 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:20.128394 28318 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.424387 28815 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.443773 25114 leader_election.cc:304] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:20.446147 29100 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:20.446704 29100 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.447063 29100 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:20.447950 23899 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.451413 23899 leader_election.cc:304] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:20.453805 29184 raft_consensus.cc:2749] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:20.454154 23899 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.455193 29184 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:20.455554 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 102099us
I20260709 12:15:20.460210 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:20.463573 28831 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.466903 28831 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.468025 23856 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:20.486287 29193 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:20.191529 28974 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.487244 29193 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.488561 29174 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:20.488873 28984 consensus_meta.cc:287] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.005s
I20260709 12:15:20.489385 28984 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.485951 29170 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:20.490837 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47558.3us
I20260709 12:15:20.508317 23857 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:20.521649 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.511564 23929 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.504081 29198 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:20.513473 29170 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.512883 28882 raft_consensus.cc:2804] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:20.530143 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:20.533421 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.533815 29181 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:20.537065 28984 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.542055 28949 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:20.538960 28980 consensus_meta.cc:287] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.726s	user 0.005s	sys 0.010s
I20260709 12:15:20.546242 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:20.559629 29212 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:20.569286 29222 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:15:20.570206 28956 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:20.570770 28956 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:20.572585 29098 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.572167 28956 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.573391 29098 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.574260 28956 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:20.576426 29098 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.545850 29184 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.580308 29184 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:20.582983 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.585507 29016 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:20.586560 29135 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.591472 29067 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.592265 23900 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:20.589965 28882 raft_consensus.cc:697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:20.595563 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:20.595337 28882 consensus_queue.cc:237] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.596304 23778 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:20.596865 28949 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:20.609318 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 60083.2us
I20260709 12:15:20.624523 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:20.625718 23931 raft_consensus.cc:2368] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14.
I20260709 12:15:20.625871 28971 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.626554 28971 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.628136 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.629696 28831 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.630527 28831 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.631533 29185 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.632732 28971 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.634233 28831 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.638299 28974 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:20.641685 23927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45606 (request call id 1464) took 1739 ms (client timeout 1530 ms). Trace:
W20260709 12:15:20.642211 23927 rpcz_store.cc:269] 0709 12:15:18.902278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.902404 (+   126us) service_pool.cc:224] Handling call
0709 12:15:20.641390 (+1738986us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002b6954. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:20.641554 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3163350784}
W20260709 12:15:20.645336 28370 consensus_meta.cc:287] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 1.162s	user 0.006s	sys 0.011s
I20260709 12:15:20.648963 29053 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.656286 29198 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.658056 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45620 (request call id 3038) took 1656 ms. Trace:
I20260709 12:15:20.664249 29184 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.665433 29052 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.666103 29052 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.669054 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:20.670086 25177 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:20.608497 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:20.606122 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 101658us
I20260709 12:15:20.671157 28318 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.672487 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:20.604919 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:20.666652 29186 raft_consensus.cc:2804] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:20.692684 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.592304 29051 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.606959 29193 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.590941 28980 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.695960 28621 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:20.698318 29052 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.698787 29193 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:20.699363 28621 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:20.699725 29193 consensus_queue.cc:1059] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:20.700822 23851 raft_consensus.cc:1240] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:20.702607 29095 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:20.703078 29095 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:20.704278 29095 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.708313 29095 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.709769 28621 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:20.710729 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.711369 28621 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.697436 28980 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:20.712567 28335 consensus_meta.cc:287] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.825s	user 0.014s	sys 0.001s
W20260709 12:15:20.695375 28884 consensus_meta.cc:287] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.879s	user 0.009s	sys 0.007s
I20260709 12:15:20.723917 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.725136 28935 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:20.725775 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.726318 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.727114 23855 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:20.727305 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.724121 28621 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:20.728197 23852 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:20.658522 23932 rpcz_store.cc:276] 0709 12:15:19.001565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.002135 (+   570us) service_pool.cc:224] Handling call
0709 12:15:19.002411 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.109805 (+107394us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:19.663124 (+553319us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.913096 (+249972us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.913322 (+   226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.913367 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:19.913425 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.657586 (+744161us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000f6ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:20.657647 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.657860 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":702258,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":2186,"raft.queue_time_us":140,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":10152,"spinlock_wait_cycles":247568384,"thread_start_us":791117,"threads_started":2,"wal-append.queue_time_us":333067}
I20260709 12:15:20.729387 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:20.729943 28935 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:20.730348 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.730443 28935 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.735056 28935 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:20.730909 28886 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:20.736485 28886 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.728101 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.738802 28886 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.731225 29051 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:20.739745 29051 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.740119 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:20.740867 23779 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:20.686551 29098 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:20.690147 29100 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:20.731289 29085 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:20.745882 29085 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:20.745988 29100 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:20.746913 29085 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.748373 29085 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.751941 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.752183 23933 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:20.754359 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:20.755409 29051 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.699857 29212 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.758328 29212 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:20.712469 29186 raft_consensus.cc:697] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:20.760917 29186 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.731946 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.764365 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.730175 23860 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 15: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:20.794358 23778 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:20.794763 23776 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:20.805325 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.728997 28949 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.759769 28884 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:20.819842 28999 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.967s	user 0.010s	sys 0.006s
I20260709 12:15:20.815815 28766 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:15:20.807819 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:20.826632 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.823976 29194 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.832240 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:20.810381 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:20.830892 23928 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:20.844410 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.731259 28956 consensus_queue.cc:1048] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:15:20.837098 23930 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:20.849779 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:20.828948 29103 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16.
I20260709 12:15:20.867077 23931 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:20.863921 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 132113us
I20260709 12:15:20.713073 28335 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.853075 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:20.871063 25172 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:20.849325 29184 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:20.853533 29193 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:20.876583 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:20.874526 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:20.835026 28999 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.896184 29193 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.912356 25174 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:20.918514 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 107342us
I20260709 12:15:20.919653 23853 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:20.920432 23824 leader_election.cc:400] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14.
I20260709 12:15:20.920766 23824 leader_election.cc:403] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:20.917328 23860 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:20.910753 29193 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:20.926580 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.927220 29004 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:20.945426 29004 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.945969 29004 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.927747 25175 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:20.931804 28882 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:20.948122 28882 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:20.936642 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:20.948951 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.937183 28884 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:20.958181 28882 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.962070 28999 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:20.941311 28335 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:20.942039 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 68224.4us
I20260709 12:15:20.964038 25114 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
W20260709 12:15:20.964600 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 95790.9us
W20260709 12:15:20.966403 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.944135 28935 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:20.913548 28980 consensus_queue.cc:1059] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:15:20.950985 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 80879.7us
I20260709 12:15:20.952204 23751 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:20.975332 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 200548us
I20260709 12:15:20.979103 25172 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:20.976374 23857 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:15:20.982227 23824 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:20.980000 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:20.983517 23899 leader_election.cc:304] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:20.986030 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:20.971545 29098 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:20.992980 25175 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:15:20.993530 23851 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.985592 29095 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.993669 28766 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:20.994979 28766 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:20.991768 28974 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:20.995759 23853 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:20.990139 29053 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:20.997668 29052 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:20.998103 29052 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:20.981743 28318 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:21.005333 29095 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.007727 29103 consensus_queue.cc:1059] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:21.008960 29103 consensus_queue.cc:1243] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:15:21.009547 29103 raft_consensus.cc:3055] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:21.009874 29103 raft_consensus.cc:740] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:21.005419 29211 raft_consensus.cc:2749] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:20.979981 28886 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14.
I20260709 12:15:20.995707 28974 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:21.010864 29103 consensus_queue.cc:260] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:20.995345 28766 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:21.023286 29103 raft_consensus.cc:3060] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.999137 29052 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:21.029771 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 116454us
I20260709 12:15:21.031302 29052 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.032123 29095 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:21.028613 28318 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:21.033685 28318 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.031315 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.036875 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:21.037627 25172 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:21.002050 29085 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.038363 29085 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.038928 23898 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.041025 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.041311 23853 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:21.037740 23930 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:21.037897 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.015424 29221 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:21.043375 23900 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.047319 23928 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.047948 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.048539 28935 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.008035 29056 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:15:21.055135 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.055645 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:21.059579 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:21.060581 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 108590us
I20260709 12:15:21.059939 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.066524 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:21.068498 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:21.052109 29211 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:21.073395 25172 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:21.065809 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:21.076193 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94173us
I20260709 12:15:21.081553 23851 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:21.063129 29098 raft_consensus.cc:2804] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:21.084618 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.087023 23857 raft_consensus.cc:2410] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:21.084148 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.077164 29211 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.108783 29211 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.109097 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.109843 23857 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:21.110966 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:21.112617 29053 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:21.113088 29053 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.113648 29053 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.093487 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:21.087093 28884 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:15:21.116650 23900 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.097708 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.117542 23930 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:21.098630 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.086586 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:21.101159 29184 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:21.105068 29085 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.106657 28766 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.154795 23860 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:21.121160 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.124177 28974 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.156996 28974 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.112985 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.158537 29051 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:21.140141 29095 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:21.134763 29194 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:21.158890 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46063.6us
I20260709 12:15:21.124342 23929 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:21.128031 28935 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:21.129066 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:21.137059 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:21.122551 29098 raft_consensus.cc:697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:21.139112 25174 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:21.140224 25172 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 15. (index mismatch)
W20260709 12:15:21.142691 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.146535 29184 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:21.165930 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.169576 23928 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:21.161599 23932 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:21.172616 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.176811 23931 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:21.180758 29133 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.181114 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.181298 29133 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.183881 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.186493 23930 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:21.188011 23779 raft_consensus.cc:2410] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:15:21.168826 28935 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.208827 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.215102 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.207966 28935 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.220362 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:21.220425 23779 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:21.220436 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.222237 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.222937 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.227816 23860 raft_consensus.cc:2368] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
I20260709 12:15:21.209184 29056 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:21.227401 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.235208 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.236871 23776 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:21.180612 29098 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.215360 29133 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:21.238413 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:21.240350 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 125024us
W20260709 12:15:21.257992 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 121082us
I20260709 12:15:21.262797 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.263572 23783 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:21.273380 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.275296 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:21.284554 23776 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:15:21.294868 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.237661 28956 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:15:21.305336 29247 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:15:21.307155 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 89241.5us
I20260709 12:15:21.309324 23751 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.310853 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.265760 29051 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.319424 23783 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:21.321287 25175 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:21.328933 28884 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:21.329493 28884 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:21.329989 28884 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:21.336244 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61066.2us
I20260709 12:15:21.337857 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:21.338781 29085 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:21.343357 28766 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:21.342613 29133 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.374760 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:21.372944 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.250125 29184 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.384680 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.391553 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.342269 29004 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.379446 28886 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:21.350693 29174 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:15:21.344633 29221 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:15:21.395679 23824 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.392537 23783 raft_consensus.cc:3060] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:15:21.400182 25173 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: [14.11-15.12]
I20260709 12:15:21.381052 28882 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:21.402531 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.403761 28984 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:21.404402 28984 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:21.405387 28984 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.400038 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:21.409044 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.410300 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.412384 28882 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:21.381692 29133 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.412891 28882 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:21.411404 23824 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.439827 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.445253 23783 raft_consensus.cc:2468] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
W20260709 12:15:21.457178 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.438954 29100 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:21.440960 29198 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:21.444516 28999 raft_consensus.cc:2804] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:21.483170 29120 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:21.416478 29212 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:15:21.486778 23857 raft_consensus.cc:2410] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:15:21.464561 29205 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:21.483064 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.478495 28980 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.451519 28886 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:21.504256 29252 log.cc:927] Time spent T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:15:21.470319 29198 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.559720 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.564273 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.466377 28318 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 12:15:21.516808 28999 raft_consensus.cc:697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:21.532310 29205 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:21.573395 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.576488 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 267227us
I20260709 12:15:21.580367 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.581065 23776 raft_consensus.cc:3060] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:21.558596 29198 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:21.594281 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 111514us
I20260709 12:15:21.584456 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.586745 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.587383 23933 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:21.587924 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.588724 28999 consensus_queue.cc:237] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:21.591729 29264 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.588016 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.592139 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.562208 29056 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.609895 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.611090 23778 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:21.612257 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.612489 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.579559 29205 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.625488 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.612005 23931 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:15:21.628832 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.629724 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:21.554011 28980 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.579108 29085 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:21.580000 29100 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:21.598510 29194 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.642874 23781 raft_consensus.cc:2368] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16.
I20260709 12:15:21.641172 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3aacc8d80dd2433b94e6d07927304b47" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.650789 23933 raft_consensus.cc:2468] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:21.652717 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.652980 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.653723 28974 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.654306 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.654488 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.654963 23779 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:21.655769 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.655469 29100 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.656380 23781 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.658850 29194 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.659371 23779 raft_consensus.cc:1240] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:21.653592 23784 raft_consensus.cc:3060] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.654229 28974 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.662462 28974 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:21.666065 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 63406.2us
I20260709 12:15:21.667796 23781 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:21.670181 23779 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:15:21.672426 29262 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:15:21.673022 23784 raft_consensus.cc:2468] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:21.673239 29133 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:21.676359 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.678474 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.653368 23782 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:21.653538 23785 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:21.569873 29211 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.704548 28935 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.705852 29004 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.707878 28886 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:21.648710 29056 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.723062 23776 raft_consensus.cc:2468] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:21.733839 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.737908 23929 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:21.739429 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:21.740192 23778 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.741472 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.741580 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:21.629268 28956 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:15:21.758742 23929 raft_consensus.cc:1275] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:21.761628 23933 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:21.776973 23781 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:21.778440 29275 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:21.779695 29258 log.cc:927] Time spent T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:15:21.780010 29265 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:21.780572 29265 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.782879 29265 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:21.785045 29261 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 12:15:21.768002 28370 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:15:21.789844 28956 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:21.790318 28980 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:21.790289 28956 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:21.796556 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.803961 23778 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
W20260709 12:15:21.760854 29193 consensus_meta.cc:287] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.820s	user 0.013s	sys 0.003s
I20260709 12:15:21.809597 23929 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:15:21.810137 23929 pending_rounds.cc:85] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Aborting all ops after (but not including) 10
W20260709 12:15:21.800686 29053 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.002s
I20260709 12:15:21.818385 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:21.723508 29085 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.728966 28884 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:21.833271 29249 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.426s	user 0.004s	sys 0.000s
I20260709 12:15:21.833599 28884 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.793315 28956 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.835023 28956 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:21.840510 23779 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:21.704550 28882 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.783442 29265 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:15:21.841749 25114 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:21.812511 29193 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.810379 23929 pending_rounds.cc:107] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:15:21.858709 23929 raft_consensus.cc:2889] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:21.844619 23933 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:21.856038 23927 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:21.872583 25173 raft_consensus.cc:1240] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:21.857331 29051 consensus_meta.cc:287] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Time spent flushing consensus metadata: real 0.517s	user 0.003s	sys 0.009s
I20260709 12:15:21.877166 23934 raft_consensus.cc:2393] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 15: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:21.878229 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.868775 29052 raft_consensus.cc:2804] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:21.891395 29052 raft_consensus.cc:697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:21.892553 29052 consensus_queue.cc:237] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.841774 29181 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:15:21.900919 28882 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:21.882416 28766 consensus_meta.cc:287] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.005s
I20260709 12:15:21.917605 29275 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:21.918176 29211 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:21.903398 29193 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.845702 29053 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:21.881348 29051 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:21.925297 23858 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:21.926108 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:21.936715 23851 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:21.909996 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.936784 23933 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 16. Current term is 17. Ops: []
W20260709 12:15:21.931696 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 264702us
W20260709 12:15:21.932394 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.935627 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:21.943842 23826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.945359 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:21.946062 23900 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.938457 23749 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.951859 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:21.952502 23860 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.954821 23776 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:21.956521 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:21.932698 29095 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:21.957142 23853 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.957147 23900 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.958330 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:21.925375 29270 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:15:21.973165 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:21.951784 29273 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:21.982385 28824 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:15:21.980501 29273 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.987063 29273 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.991103 29085 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.991750 23777 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:21.992888 28884 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:21.975572 29263 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:21.994004 29263 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:21.994403 29263 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:21.996549 28935 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.999022 29194 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:21.999219 23860 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:22.013502 29103 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:21.974598 29095 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.021947 29095 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:21.960733 29198 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:21.978053 29098 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:22.037328 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:22.054112 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.037736 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:22.039274 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 94016.3us
W20260709 12:15:22.028121 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 273980us
I20260709 12:15:22.098052 23853 raft_consensus.cc:2410] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:22.106208 25168 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:21.973814 29280 raft_consensus.cc:2764] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:22.109086 25175 raft_consensus.cc:3060] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.961858 29221 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:15:22.145650 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:22.043258 28886 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:22.146471 23858 raft_consensus.cc:3060] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:22.150066 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:22.049952 29264 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.033365 28935 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.044226 28766 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:22.034658 29085 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:22.042706 29053 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:22.148376 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:22.041234 29284 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:22.039532 29056 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.108924 29098 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:22.160594 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 203580us
I20260709 12:15:22.178385 25175 raft_consensus.cc:2468] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:22.162351 23853 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:22.027477 29205 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.150398 28886 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:22.144765 29181 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:22.160442 23751 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.181788 23898 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:22.191349 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 222200us
W20260709 12:15:22.191501 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 187066us
I20260709 12:15:22.193403 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.180646 29098 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:22.167542 29284 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:22.181226 29270 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:22.218945 23934 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:22.156853 29275 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:15:22.221160 23900 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.224306 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:22.234983 23927 raft_consensus.cc:2368] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15.
I20260709 12:15:22.230643 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:22.236171 23928 raft_consensus.cc:2393] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:22.232935 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:22.164482 29266 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:15:22.236624 25175 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.172606 28935 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.239189 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:22.249490 23851 raft_consensus.cc:2393] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 16: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:22.220877 29004 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:22.247377 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:22.255806 23930 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 12: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:22.195497 29053 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.216269 29184 raft_consensus.cc:2804] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:22.260984 25175 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:22.182303 29085 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.259578 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:22.262573 23929 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:22.266368 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.264626 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:22.271171 23860 raft_consensus.cc:2410] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:15:22.266858 23853 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.266841 23778 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:22.272626 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 123339us
W20260709 12:15:22.273820 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 55573.1us
I20260709 12:15:22.275074 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:22.281949 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 166458us
W20260709 12:15:22.282330 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.222488 29284 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.276120 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.287130 23783 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.276253 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.247200 29193 raft_consensus.cc:2804] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:22.255342 29004 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.290158 23749 leader_election.cc:400] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15.
I20260709 12:15:22.295222 23749 leader_election.cc:403] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:22.285674 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.248353 29270 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:22.297544 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 145621us
I20260709 12:15:22.301326 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.303367 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:22.275671 29280 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.310460 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.312057 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:22.312541 29288 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:15:22.313540 29290 log.cc:927] Time spent T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:15:22.314402 29292 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:15:22.306991 23857 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:22.316704 25168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44500 (request call id 741) took 1202 ms. Trace:
W20260709 12:15:22.328050 29291 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:15:22.329121 29205 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:22.300674 29004 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.309278 25177 raft_consensus.cc:2955] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:22.331493 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.332270 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.276276 29184 raft_consensus.cc:697] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:22.341897 23783 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:22.342922 23853 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:22.344693 23858 raft_consensus.cc:2468] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
W20260709 12:15:22.357954 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 67957.1us
I20260709 12:15:22.388969 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.392831 23782 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:22.347448 25168 rpcz_store.cc:276] 0709 12:15:21.114420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.115896 (+  1476us) service_pool.cc:224] Handling call
0709 12:15:22.316266 (+1200370us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b5400492854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:22.316586 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727276288}
I20260709 12:15:22.359078 23780 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:22.362666 29273 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.365093 29085 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.366729 28935 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:22.368211 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:22.370213 29263 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.388576 23778 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:22.337958 29264 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:22.404523 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 109905us
I20260709 12:15:22.372455 29274 raft_consensus.cc:3060] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.344698 29280 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.409821 29205 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:22.414186 29280 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.419737 29299 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:22.420459 29299 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.422770 29299 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.427845 25173 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
W20260709 12:15:22.431879 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 151017us
I20260709 12:15:22.432786 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.433337 29284 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.434584 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:22.334820 29270 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.411414 29266 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:15:22.308768 28886 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:22.436491 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47616.3us
I20260709 12:15:22.440631 23898 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.438087 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.439543 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:22.454070 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:22.465521 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.436088 29198 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.469099 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:22.469835 23930 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:22.439360 28370 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:22.439710 29282 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:22.483872 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:22.485244 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:22.454922 29264 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:22.488343 23899 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.489873 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:22.491204 23899 leader_election.cc:400] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16.
I20260709 12:15:22.441624 28935 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:22.493606 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55612.5us
I20260709 12:15:22.502506 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:22.435554 29098 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.479068 29299 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.511231 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.512101 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.488413 29052 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:22.493350 29194 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.467149 28766 consensus_queue.cc:1048] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:15:22.505795 29273 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.514400 29198 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:22.510576 29211 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.436595 29184 consensus_queue.cc:237] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.537431 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.522924 29198 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.522377 29299 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.523212 29056 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:22.508289 28935 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.548190 29294 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:22.543376 29193 raft_consensus.cc:697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:22.556663 29198 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:22.572305 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58957.5us
I20260709 12:15:22.574193 25177 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: [13.14-14.15]
I20260709 12:15:22.567847 29299 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:15:22.564980 29303 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:22.564954 29282 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:22.588253 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43930.6us
I20260709 12:15:22.560269 28621 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:15:22.570063 29088 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:22.595314 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:22.599984 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.600718 23782 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.601529 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.602998 29193 consensus_queue.cc:237] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.603852 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.603940 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.605195 23780 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:22.589183 29205 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:22.612149 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.614720 23778 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:22.619042 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:22.619963 25169 raft_consensus.cc:2393] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:22.620527 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.621917 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.622673 29051 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.623463 29051 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:22.618930 29302 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:15:22.614468 29273 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:15:22.613729 29205 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:22.641794 29282 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.642271 29205 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.15, Last appended by leader: 15, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.643358 29205 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.644335 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.630976 29004 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.646850 23782 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:22.623268 23751 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.609494 29088 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.649996 29004 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.650486 25172 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:22.614329 28935 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.651903 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:22.652597 29284 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.650173 29088 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.658749 29266 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:22.659237 29319 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.659327 29266 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.659962 29319 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.661804 29266 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:22.662096 29319 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:22.672174 23749 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:22.674108 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:22.677294 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82057.6us
I20260709 12:15:22.680909 23784 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:22.666383 29303 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:22.684687 28824 consensus_queue.cc:1059] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:22.685936 28824 raft_consensus.cc:3055] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:22.686321 28824 raft_consensus.cc:740] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:22.676582 29056 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:15:22.688159 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 44763.4us
I20260709 12:15:22.687275 28824 consensus_queue.cc:260] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.688576 23785 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:22.689749 29307 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:22.690997 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.691699 23778 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:22.692843 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.698997 29211 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.689376 28824 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.680857 29280 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:22.694829 29303 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.683686 29284 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:22.704278 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:22.704926 23899 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.674647 29098 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.712525 29273 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:15:22.719627 29211 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:22.725989 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.726908 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.745100 23778 raft_consensus.cc:2393] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:22.727571 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.728153 29307 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:22.730468 28935 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.731679 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:22.737893 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:22.738397 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:22.743059 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:22.744565 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.748522 23782 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:22.750535 23780 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:22.751889 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.756520 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.756826 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:22.758597 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:22.760528 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:22.761319 23782 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:22.769865 23785 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:22.773329 23925 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:22.711477 29303 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:22.744499 28956 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:22.784835 23782 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:22.775705 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.754796 29284 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:22.789222 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 166076us
I20260709 12:15:22.788489 29098 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:22.766139 29212 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:22.765576 29211 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:22.795657 29184 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.783303 28956 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.759696 28935 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.835546 28935 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:22.812203 29211 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.789528 29315 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:15:22.809677 29212 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:22.826495 29284 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:22.829207 28956 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:22.848239 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.794798 29274 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15.
W20260709 12:15:22.850109 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 74510.2us
I20260709 12:15:22.801774 29098 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.852501 29098 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.853003 23860 raft_consensus.cc:1275] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:15:22.856961 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48284.5us
I20260709 12:15:22.857843 23751 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.854733 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:22.870515 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.871201 23782 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:15:22.875118 23899 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:22.876892 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.881060 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.890123 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:22.895112 23776 raft_consensus.cc:2393] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:22.876199 28884 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.870685 29270 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.878340 29273 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.870205 28766 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:15:22.884343 29088 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.889945 29307 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:22.919166 23751 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.890429 28935 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:15:22.931104 23751 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:22.922667 29184 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:22.958511 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61545.4us
I20260709 12:15:22.932924 28882 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:22.928925 29307 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:22.971859 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 90905us
W20260709 12:15:22.974910 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.972918 25113 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:22.976805 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45792.6us
I20260709 12:15:22.935047 29051 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:22.923882 29273 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.960183 29193 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.925782 29280 consensus_queue.cc:1059] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:15:22.963467 29264 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.917900 29303 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:22.972251 29307 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:23.002260 25114 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:23.003021 23751 leader_election.cc:400] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
I20260709 12:15:22.916805 29212 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:23.004738 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 86104.8us
I20260709 12:15:23.005717 23751 leader_election.cc:403] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:23.009857 29294 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:23.010485 29294 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:22.989679 29211 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.028244 23784 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:15:22.974145 29316 consensus_queue.cc:1048] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:15:22.971580 29184 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.006806 28882 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.007057 28621 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:23.036562 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.037729 29273 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.039062 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.039965 29264 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:23.042498 23777 raft_consensus.cc:2468] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:23.044641 29184 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:23.052455 29327 log.cc:927] Time spent T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:15:23.013197 29280 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16.
I20260709 12:15:23.059549 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.060873 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.062453 23780 raft_consensus.cc:1240] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:23.002216 29193 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.071646 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.072461 23851 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:23.075299 23857 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:23.032317 29004 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:23.081471 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.082417 23857 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:23.011785 29314 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:15:23.086385 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:23.033423 29211 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.096483 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.097566 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:23.101800 23900 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:23.081327 29100 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.066136 28621 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.080368 29004 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.092511 28824 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.121467 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.090662 29280 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:23.109688 29056 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:23.099862 29184 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.119356 23857 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:15:23.144532 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:23.119956 29194 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:23.120100 29004 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.173012 29345 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:23.173569 29345 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:23.184629 29345 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:23.125272 28824 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.119750 29100 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:23.201658 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 55182.3us
I20260709 12:15:23.169955 29184 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.131868 29280 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.191840 29098 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.188865 29095 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:15:23.216553 29308 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.220247 29308 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.222533 29308 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.204528 29303 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.228119 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.249049 23778 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:23.206866 29053 raft_consensus.cc:493] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:23.232630 29345 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.262920 29270 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.258656 29053 raft_consensus.cc:515] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.264510 29345 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.265647 28621 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.262118 29193 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.275930 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.281570 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.314841 23778 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:23.318473 23900 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:23.314785 29193 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.320616 29193 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.266124 29053 leader_election.cc:290] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:23.267673 29336 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 12:15:23.271026 29198 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.691s	user 0.007s	sys 0.002s
I20260709 12:15:23.285233 28886 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:23.321796 29298 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.953s	user 0.004s	sys 0.000s
I20260709 12:15:23.333351 29056 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:23.268143 29211 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.273288 29100 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.324275 29308 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:23.344751 29308 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:23.345176 23777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 3037) took 1067 ms. Trace:
I20260709 12:15:23.345773 29308 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.346903 29275 raft_consensus.cc:3055] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:23.347311 29275 raft_consensus.cc:740] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:23.348235 29275 consensus_queue.cc:260] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.351985 29275 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.283519 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.267525 28956 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:23.287449 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.342175 29004 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.330811 29280 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.343199 29198 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.276490 29263 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.330387 28884 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.316675 29345 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.322213 29194 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.352965 29294 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.347384 29193 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:15:23.338146 29205 consensus_meta.cc:287] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.000s
I20260709 12:15:23.333526 29184 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.345577 23777 rpcz_store.cc:276] 0709 12:15:22.277253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.331205 (+ 53952us) service_pool.cc:224] Handling call
0709 12:15:23.344927 (+1013722us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b54001ee6d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:23.345081 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102539136}
I20260709 12:15:23.364738 23857 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:23.365427 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.385656 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.374114 23860 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:15:23.375828 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.312537 29307 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.367610 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:23.388053 23852 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:23.386620 23853 raft_consensus.cc:1240] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:15:23.388536 23851 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:23.389130 23857 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:23.391815 23857 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:23.386405 23860 pending_rounds.cc:85] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 9
I20260709 12:15:23.396705 23860 pending_rounds.cc:107] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:15:23.397013 23860 raft_consensus.cc:2889] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:23.405429 23857 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:23.409094 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.409780 23856 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:23.381120 29345 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.383870 29194 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.428133 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.441072 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.445747 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:23.446831 23856 raft_consensus.cc:2410] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 15 because replica has last-logged OpId of term: 14 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 14.
I20260709 12:15:23.444640 23852 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 10.
I20260709 12:15:23.448498 23858 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:23.450706 23857 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:15:23.451112 29103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.105s	user 0.013s	sys 0.008s
I20260709 12:15:23.451941 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:23.456099 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 76414us
I20260709 12:15:23.466650 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.471529 23860 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:23.477106 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.477206 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.478976 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.479868 23779 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:23.480959 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.483227 23777 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:23.436574 28956 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.485725 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:23.487175 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.485975 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.489372 23780 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:23.470463 29103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.013s	sys 0.008s
W20260709 12:15:23.482287 29277 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.001s	sys 0.020s
I20260709 12:15:23.499300 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:23.503400 29326 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.588s	user 0.002s	sys 0.001s
I20260709 12:15:23.500111 29198 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.515463 23779 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:23.525408 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:23.526033 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.501572 29277 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.001s	sys 0.020s
I20260709 12:15:23.527403 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.529582 23860 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:15:23.547122 23860 pending_rounds.cc:85] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 11
I20260709 12:15:23.533504 29355 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.534684 23780 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:23.535997 29194 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.536442 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.536675 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:23.537379 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.537535 23858 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 15. Preceding OpId from leader: term: 15 index: 15. (term mismatch)
I20260709 12:15:23.538245 29307 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.539021 23932 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1494) took 1294 ms. Trace:
I20260709 12:15:23.554181 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:23.538407 23854 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:23.540723 29100 consensus_queue.cc:1059] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:23.541605 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.543923 23933 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:23.539325 29311 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.918s	user 0.003s	sys 0.000s
W20260709 12:15:23.544871 29320 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.927s	user 0.003s	sys 0.000s
I20260709 12:15:23.540743 29345 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.547227 23779 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 18: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:23.547418 23860 pending_rounds.cc:107] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:15:23.564767 23860 raft_consensus.cc:2889] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:23.547724 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1497) took 1261 ms. Trace:
I20260709 12:15:23.550272 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:23.553561 23858 pending_rounds.cc:85] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Aborting all ops after (but not including) 14
I20260709 12:15:23.565817 23858 pending_rounds.cc:107] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: Aborting uncommitted NO_OP operation due to leader change: 14.15
I20260709 12:15:23.566069 23858 raft_consensus.cc:2889] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:23.554389 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:23.534734 29294 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.567095 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.567459 25168 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:23.554438 23932 rpcz_store.cc:276] 0709 12:15:22.244223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.256873 (+ 12650us) service_pool.cc:224] Handling call
0709 12:15:22.273976 (+ 17103us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:22.294273 (+ 20297us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:22.310448 (+ 16175us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.894232 (+583784us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800102760. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b 00007fb2ad351d23 00007fb2ad35223a 00007fb2ad7bca2a
0709 12:15:23.001658 (+107426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.001712 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:23.002714 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.538793 (+536079us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b540038fc54. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:23.538848 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.538965 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":407,"prepare.queue_time_us":15996,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":2147,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":6451,"spinlock_wait_cycles":50990336,"thread_start_us":107168,"threads_started":1,"wal-append.queue_time_us":720397}
I20260709 12:15:23.581914 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:23.587406 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.587357 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.588112 23781 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.565486 28824 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.576669 29198 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:23.593087 29319 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:15:23.595760 28999 consensus_queue.cc:1059] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:23.579242 29004 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.604907 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.606819 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.554616 29088 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:15:23.591300 29273 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:23.586453 28621 raft_consensus.cc:3055] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:23.618196 28621 raft_consensus.cc:740] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:23.619320 23926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45606 (request call id 1495) took 1363 ms. Trace:
I20260709 12:15:23.619261 28621 consensus_queue.cc:260] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.586287 29205 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:23.619864 23926 rpcz_store.cc:276] 0709 12:15:22.255955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.256146 (+   191us) service_pool.cc:224] Handling call
0709 12:15:22.274592 (+ 18446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:22.285125 (+ 10533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:22.285158 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:22.306649 (+ 21491us) log.cc:844] Serialized 26 byte log entry
0709 12:15:22.545056 (+238407us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:22.545101 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.461607 (+916506us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54002b7354. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:23.469200 (+  7593us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.538639 (+ 69439us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:23.619045 (+ 80406us) raft_consensus.cc:1685] finished
0709 12:15:23.619072 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.619248 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28526,"raft.queue_time_us":5959,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7251,"spinlock_wait_cycles":55486592,"thread_start_us":212013,"threads_started":1,"wal-append.queue_time_us":288430}
I20260709 12:15:23.622478 23777 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:23.555810 29355 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.627868 25173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53494 (request call id 538) took 1189 ms. Trace:
I20260709 12:15:23.628304 25173 rpcz_store.cc:276] 0709 12:15:22.438574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.439757 (+  1183us) service_pool.cc:224] Handling call
0709 12:15:22.440138 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:22.440590 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:22.546763 (+106173us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.595068 (+ 48305us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:22.608159 (+ 13091us) log.cc:844] Serialized 52 byte log entry
0709 12:15:22.608338 (+   179us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:22.613343 (+  5005us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.613645 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.613729 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:23.541063 (+927334us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2ad34d86e 00007fb2ad34cbb6 00007fb2ad34c7d8 00007fb2ad356221 00007fb2ad3561d9 00007fb2ad356169
0709 12:15:23.627565 (+ 86502us) raft_consensus.cc:1685] finished
0709 12:15:23.627608 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.627776 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":869,"prepare.queue_time_us":216230,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":4613,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":34074,"spinlock_wait_cycles":39890176,"thread_start_us":127033,"threads_started":2,"wal-append.queue_time_us":57019}
I20260709 12:15:23.634722 29355 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.637158 28999 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:23.637686 28999 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:23.639111 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 85607.3us
I20260709 12:15:23.575671 23934 rpcz_store.cc:276] 0709 12:15:22.286557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.287980 (+  1423us) service_pool.cc:224] Handling call
0709 12:15:22.303476 (+ 15496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:22.303724 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:22.445383 (+141659us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.544581 (+ 99198us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.655703 (+111122us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b180044aa60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2b168658f 00007fb2aa6c5571 00007fb2aa6c77c1 00007fb2aa6c7779 00007fb2aa6c7709 00007fb2aa6c76d1 00007fb2aa6c69cd
0709 12:15:23.494545 (+838842us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b180044aa60. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa6a7e07 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c2455 00007fb2ad7aa060 00007fb2ad7bc716 00007fb2ad7cd287 00007fb2ad7cd208
0709 12:15:23.522545 (+ 28000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:23.522591 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:23.522751 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.522813 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:23.547659 (+ 24846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99016,"prepare.queue_time_us":27486,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":95,"raft.queue_time_us":58034,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":15681,"spinlock_wait_cycles":6020608,"thread_start_us":239152,"threads_started":3,"wal-append.queue_time_us":113326}
I20260709 12:15:23.647579 29319 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:23.648065 29319 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:23.646510 23824 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:23.658108 23751 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.660064 23781 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:23.661392 25173 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:23.661636 23751 leader_election.cc:400] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:23.662798 23751 leader_election.cc:403] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:23.664319 23751 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:23.666067 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:23.659565 29198 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:23.667703 29198 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.669081 29263 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:23.664975 29280 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.670912 23751 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:23.662388 29270 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:23.671913 29263 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:23.672434 29263 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:23.673337 29263 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.662963 29265 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:15:23.666036 29299 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:23.676748 29299 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.677125 29299 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:23.683341 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.687945 29181 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:23.688491 29181 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:23.638620 28999 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.689415 29181 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.693354 28999 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.657104 29275 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:23.715116 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:23.715963 25173 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:23.323235 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:23.727893 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 176159us
I20260709 12:15:23.740147 25173 raft_consensus.cc:1275] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:23.700946 29362 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 12:15:23.709987 29133 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.014s	sys 0.002s
W20260709 12:15:23.752377 29133 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.386s	user 0.015s	sys 0.002s
I20260709 12:15:23.759030 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:23.760462 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:23.762140 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.703715 28956 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:15:23.733970 29265 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:15:23.764492 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:23.841132 29299 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.843770 28956 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:23.844456 28956 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.846617 28956 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.853071 29280 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13.
I20260709 12:15:23.768232 29051 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:15:23.854826 29053 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:23.855222 29053 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:23.856110 29053 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:23.869890 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 114047us
I20260709 12:15:23.886606 25173 raft_consensus.cc:1240] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:23.893887 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.897863 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:23.843518 29299 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.882218 29100 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.758725 29274 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:23.908206 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:23.914971 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.916739 25173 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:23.916683 28370 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:23.921106 28370 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:23.922124 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.757450 29344 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.165s
I20260709 12:15:23.926564 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:23.915285 29100 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.924664 29274 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:23.939914 29319 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.940644 29274 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.944832 23853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46448 (request call id 806) took 1095 ms. Trace:
I20260709 12:15:23.938160 29221 consensus_queue.cc:1059] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:15:23.947862 28370 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:23.945195 29100 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:23.960965 29100 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.961506 29100 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.964018 29100 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:23.968533 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 170) took 6114 ms. Trace:
I20260709 12:15:23.973938 29319 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:23.974728 29304 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
W20260709 12:15:23.975826 23823 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.968678 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 548779us
I20260709 12:15:23.945253 23853 rpcz_store.cc:276] 0709 12:15:22.849452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.849611 (+   159us) service_pool.cc:224] Handling call
0709 12:15:22.850015 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:23.388657 (+538642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:23.944517 (+555860us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400412dd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:23.944740 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233476992}
I20260709 12:15:23.979522 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.981335 23931 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:23.990461 25173 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:23.995182 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:23.979976 29103 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50860 (request call id 1) took 4199 ms. Trace:
I20260709 12:15:23.980147 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.000408 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:24.002853 23929 raft_consensus.cc:1240] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:15:24.003984 23926 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.005976 23933 raft_consensus.cc:1240] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:24.006578 23934 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:24.008167 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.008989 23928 raft_consensus.cc:2368] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15.
I20260709 12:15:23.999136 29103 rpcz_store.cc:276] 0709 12:15:19.780290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.780696 (+   406us) service_pool.cc:224] Handling call
0709 12:15:20.056010 (+275314us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b54002736a0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad72f520 00007fb2ad7b8b42 00007fb2ad7b7800 00007fb2ad7b09fa 00007fb2b27784c9 00007fb2abcf4c54 00007fb2abcf4be2 00007fb2abcf4b11 00007fb2abcf4a8c 00007fb2abcf3d42
0709 12:15:20.509439 (+453429us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b5800795ed0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad70e0aa 00007fb2ad790447 00007fb2ad7ad70a 00007fb2ad7ae76a 00007fb2ad7ad91b 00007fb2ad7b903c 00007fb2ad7b86b3 00007fb2ad7b09fa 00007fb2b27784c9 00007fb2abcf4c54
0709 12:15:20.587021 (+ 77582us) log.cc:844] Serialized 494 byte log entry
0709 12:15:20.964208 (+377187us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b180028e4c0. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2b358dab1 00007fb2b359a281 00007fb2aa63fc0a 00007fb2b3d7f219 00007fb2aa6c7a40 00007fb2aa6c24e4 00007fb2aa6c2455 00007fb2ad70bb0b 00007fb2ad790b63 00007fb2ad7b8752 00007fb2ad7b09fa
0709 12:15:23.979844 (+3015636us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57590,"raft.queue_time_us":124259,"raft.run_cpu_time_us":1712,"raft.run_wall_time_us":76273,"spinlock_wait_cycles":55437312,"thread_start_us":261160,"threads_started":1,"wal-append.queue_time_us":314607,"wal-append.run_cpu_time_us":2778,"wal-append.run_wall_time_us":1056710}
I20260709 12:15:24.018491 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.019153 23934 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:24.019851 29349 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.020474 29345 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:24.020799 29211 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.021015 29345 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.021421 29211 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:24.023258 29345 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.024583 29318 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.028023 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:23.977375 29100 raft_consensus.cc:3055] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:24.032343 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.029428 23899 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.042780 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:24.043519 23925 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:23.975358 29304 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:23.969244 23683 rpcz_store.cc:276] 0709 12:15:17.854036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.854192 (+   156us) service_pool.cc:224] Handling call
0709 12:15:23.968303 (+6114111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:20.137667 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:20.138257 (+   590us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:20.138297 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:15:20.138309 (+    12us) tablet.cc:662] Decoding operations
0709 12:15:20.183403 (+ 45094us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:20.192670 (+  9267us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:20.193951 (+  1281us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:15:20.199829 (+  5878us) tablet.cc:701] Row locks acquired
0709 12:15:20.199849 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:15:20.200004 (+   155us) write_op.cc:270] Start()
0709 12:15:20.200120 (+   116us) write_op.cc:276] Timestamp: P: 1783599320199954 usec, L: 0
0709 12:15:20.200139 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:15:20.200863 (+   724us) log.cc:844] Serialized 7432 byte log entry
0709 12:15:20.691636 (+490773us) op_driver.cc:464] REPLICATION: finished
0709 12:15:20.953673 (+262037us) write_op.cc:301] APPLY: starting
0709 12:15:20.953816 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:21.312653 (+358837us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:21.312808 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:21.671925 (+359117us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:21.672341 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:15:21.672406 (+    65us) write_op.cc:312] APPLY: finished
0709 12:15:21.672968 (+   562us) log.cc:844] Serialized 172 byte log entry
0709 12:15:21.897582 (+224614us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:21.897911 (+   329us) write_op.cc:454] Released schema lock
0709 12:15:21.897988 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:21.898465 (+   477us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:21.898497 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1182868,"reported_tablets":25,"spinlock_wait_cycles":1792,"child_traces":[["op",{"apply.queue_time_us":222808,"apply.run_cpu_time_us":40606,"apply.run_wall_time_us":949630,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"mutex_wait_us":7629,"num_ops":19,"prepare.queue_time_us":297894,"prepare.run_cpu_time_us":9515,"prepare.run_wall_time_us":75160,"raft.queue_time_us":200883,"raft.run_cpu_time_us":32868,"raft.run_wall_time_us":181148,"replication_time_us":491208,"thread_start_us":595958,"threads_started":5,"wal-append.queue_time_us":389178,"wal-append.run_cpu_time_us":33159,"wal-append.run_wall_time_us":2402419}]]}
I20260709 12:15:24.046661 23898 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.045336 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:24.050530 23900 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:24.057680 23932 raft_consensus.cc:2393] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 18: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:24.062019 23929 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:24.027508 29095 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:15:24.062454 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.039290 29100 raft_consensus.cc:740] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.069101 23933 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 17: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:24.058298 29211 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:24.033195 29307 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:15:24.020403 29349 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.077950 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:24.082046 23929 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:24.065487 29194 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:24.071979 29205 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.033334 29294 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:24.079620 29085 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:24.062012 29056 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:24.085290 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.088960 23933 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
W20260709 12:15:24.092213 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 176140us
I20260709 12:15:24.092377 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0370e122479081184511cb1224f1" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.094522 25173 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:24.064427 29345 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:15:24.045243 29318 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.098276 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.083854 29194 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.100428 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.101289 25173 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:24.102633 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.105329 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.106057 23857 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:24.089722 29056 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.109477 23857 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:15:24.095564 29085 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:24.120560 29085 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.102176 29194 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:24.129164 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.129909 29053 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:24.090340 29205 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.115568 25168 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:24.117175 29318 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.103348 29100 consensus_queue.cc:260] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.117815 29181 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.110522 29056 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.111313 23824 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.148806 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.150488 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:24.150775 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.152535 29280 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:24.153003 29280 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.153416 29280 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.156512 29344 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:24.157621 29344 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:24.136636 23925 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:24.157842 23749 leader_election.cc:304] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.162138 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.170506 29098 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.158079 29344 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:24.174410 29344 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.175868 29344 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.166270 29211 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:24.104588 29103 consensus_queue.cc:1048] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:15:24.157534 29304 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.186239 28824 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.177059 29274 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:24.186877 28824 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.188910 29265 raft_consensus.cc:2749] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.159556 29334 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.191644 29280 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.193001 25175 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:24.195472 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.197585 23925 raft_consensus.cc:1275] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:24.209321 23929 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
W20260709 12:15:24.213084 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 306315us
I20260709 12:15:24.217772 23926 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:24.156668 29221 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:24.189200 28824 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:24.238893 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 174758us
I20260709 12:15:24.214435 29205 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.191474 29334 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:24.243346 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 213990us
I20260709 12:15:24.234356 29319 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.225395 29198 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:15:24.267904 23857 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:24.284024 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.270190 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.274456 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.249488 29349 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.285379 25175 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:24.276093 29334 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.275257 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.277773 29375 consensus_queue.cc:1059] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:24.292094 29375 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:24.292526 29375 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.293332 29375 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.294634 29375 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:24.278687 29379 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.297888 29349 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:24.298432 29379 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.268797 29280 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.300472 29221 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.301486 29221 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.280838 29319 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.304297 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.305255 29085 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:24.305680 29085 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.306061 29085 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:24.306445 29369 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.201s	user 0.003s	sys 0.001s
I20260709 12:15:24.246027 29100 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.305274 29319 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.315546 29085 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.316637 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.317515 23932 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:24.318032 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.318557 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.319305 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.319811 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.318715 23931 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:24.320588 23934 raft_consensus.cc:2410] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:15:24.321512 29085 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.321954 29318 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:24.304560 29051 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:24.321523 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:24.323419 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.322266 29085 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.324659 29085 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:24.326198 29085 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.327060 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:24.328739 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:24.329550 29345 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:24.329903 23825 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.330102 29345 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:24.322405 29318 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.330894 29318 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:24.331791 29319 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:24.332835 25173 raft_consensus.cc:2393] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:24.332089 29273 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.336436 29273 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:24.337054 29273 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.338025 29273 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.339164 29337 consensus_queue.cc:1059] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:24.340204 29337 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:24.340608 29337 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:24.341498 29337 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.343053 29337 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.320508 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.343979 29194 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:24.350435 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.354298 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:24.355401 29362 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:24.322988 29051 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:24.355923 29362 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.356420 29362 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:24.357156 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.359710 29051 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.362540 29299 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:24.365950 23824 leader_election.cc:400] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15.
I20260709 12:15:24.366380 23824 leader_election.cc:403] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:24.366742 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.367792 29379 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.331039 29345 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.370213 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.375202 29194 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.339409 28621 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:24.380751 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.385226 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.388594 29265 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.387856 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.389030 29056 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.400821 29373 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:24.389210 29265 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.381307 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:24.405122 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.414044 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.415830 29373 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:24.416730 29373 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.419871 29274 raft_consensus.cc:493] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.413513 29265 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.420509 29274 raft_consensus.cc:515] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.422695 29274 leader_election.cc:290] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.423219 29375 consensus_queue.cc:1048] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:24.387905 29299 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.410033 23678 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.426986 23678 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.428848 23851 raft_consensus.cc:2468] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:24.433874 29056 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.429294 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:24.431182 29274 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.434691 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.435074 29274 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.435745 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:24.437285 29274 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.319169 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:24.444821 23930 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:24.439148 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.464022 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.464145 23824 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.464708 23933 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:24.468451 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:24.470654 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.476714 23932 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:24.479092 25119 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:24.463537 28621 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:15:24.470534 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 420088us
I20260709 12:15:24.463397 29375 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:24.474084 28999 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:24.481436 28886 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:24.499248 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.484536 29133 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:24.491304 28370 raft_consensus.cc:2804] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:24.505340 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.472083 29373 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:15:24.482506 29362 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:24.597041 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 88235.6us
I20260709 12:15:24.597999 23900 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.599426 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.437815 23678 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 10 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.496065 28999 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.487164 29205 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:15:24.601799 23900 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.506784 28882 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:24.497880 28956 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 12:15:24.605686 29269 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:24.605036 29194 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:24.606701 29382 raft_consensus.cc:2749] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:24.636485 23859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.022s	sys 0.034s
I20260709 12:15:24.613387 28999 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:24.646914 23859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.022s	sys 0.034s
I20260709 12:15:24.612097 29265 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.617422 28370 raft_consensus.cc:697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:24.619684 28886 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.648612 23932 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:24.611619 29344 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.658879 23853 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:24.659277 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.675227 23856 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:24.659675 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.644894 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet de3c70defd864769b26b52b7f36548cd with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:24.682632 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.683571 23860 raft_consensus.cc:2393] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:24.638777 29383 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:24.641572 28766 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:24.651260 29212 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:24.647610 29265 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:24.700472 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 416492us
W20260709 12:15:24.706696 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 440210us
I20260709 12:15:24.714288 28766 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.714993 28766 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.738092 29304 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:24.738787 29304 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.743359 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:24.743854 29212 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:24.744347 29212 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.745373 29304 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.709467 23749 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:24.694221 29366 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.679327 28886 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.679688 29318 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:24.769745 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 60363.4us
I20260709 12:15:24.684547 29344 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.776598 28621 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.792112 23929 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:24.768913 29274 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.769163 29275 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.808585 23853 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:24.810128 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.811563 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.813566 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.766536 23678 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.804855 29274 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.812229 29275 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:24.774595 29366 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.832231 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:24.833056 23934 raft_consensus.cc:606] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.833244 29396 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:24.833451 23934 raft_consensus.cc:614] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:24.824059 28621 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:24.836133 29385 log.cc:927] Time spent T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:15:24.837033 29299 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) with higher term. Message: Invalid argument: T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15.
I20260709 12:15:24.823889 29270 raft_consensus.cc:2749] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:24.841034 29270 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:24.841903 29265 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.842516 29265 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:24.842839 29265 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:24.844822 29318 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:24.844149 23678 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 10 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.846738 29344 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.825299 29390 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.848997 29396 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.853070 29053 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.853646 29053 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.813071 29263 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:24.857470 29263 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.851397 29396 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.858218 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.858623 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.859081 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.859741 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.860327 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.860955 23859 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:24.864549 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:24.867267 29396 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:15:24.871048 29337 consensus_meta.cc:287] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.002s
I20260709 12:15:24.846275 23678 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.873332 23678 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:24.875312 23678 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:24.876502 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:24.817117 29386 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.313s	user 0.001s	sys 0.001s
I20260709 12:15:24.877316 23900 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:24.885248 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:24.785369 29212 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.887104 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:24.880509 23857 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:24.849080 29390 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.798602 28370 consensus_queue.cc:237] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.881280 29318 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:15:24.930789 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.931537 23858 raft_consensus.cc:2393] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 15: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:24.932416 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.933521 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.934554 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.935120 23858 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:24.886376 29387 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:24.928376 29269 consensus_queue.cc:1059] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:24.940008 29269 raft_consensus.cc:3055] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:24.940461 29269 raft_consensus.cc:740] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:24.941318 29269 consensus_queue.cc:260] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:24.942632 29269 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.945163 23927 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:24.897480 29053 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.930025 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.921635 29382 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.951757 23857 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:24.953985 29274 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.928773 23854 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:24.915249 29362 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.958212 28766 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.878615 29265 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:24.913106 29304 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:24.967078 29366 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.896312 29337 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.892241 29056 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:24.926853 29378 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:24.970212 29212 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:24.888108 29263 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.891201 23856 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:24.977351 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:24.986691 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:24.968643 29275 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:24.971134 28886 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.002421 23930 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:15:25.004165 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 232795us
W20260709 12:15:25.005884 25170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.000s	sys 0.009s
I20260709 12:15:25.005774 29265 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:25.008040 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:25.053376 25170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.002s	sys 0.009s
I20260709 12:15:25.057904 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:25.016974 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:25.059085 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.060541 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.062114 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:25.064878 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 50722.4us
I20260709 12:15:25.004403 29056 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:25.068348 25169 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
W20260709 12:15:25.057403 29375 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.016s	sys 0.008s
I20260709 12:15:25.069515 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:25.069132 23931 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:15:25.071346 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.003639 29304 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:25.002358 29053 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.997177 29294 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:15:25.051115 29299 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.004762 29378 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:25.077560 23857 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:25.073426 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.077489 23934 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 17: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:25.083453 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77aa2cbe928a45d4b1652ca10a80aa99" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.084362 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:25.074729 29375 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.016s	sys 0.008s
I20260709 12:15:25.082948 29275 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:25.074028 29056 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:25.117411 23777 raft_consensus.cc:1275] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:25.117743 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:25.038022 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 532723us
I20260709 12:15:25.012502 29280 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.078306 29304 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:25.073083 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.113065 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:25.079922 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 93329.5us
I20260709 12:15:25.107146 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.119194 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.077217 29211 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:25.121343 23855 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:25.122002 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:24.982137 29382 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:25.127441 23750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.069097 28886 consensus_queue.cc:1059] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:15:25.087286 29051 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.167392 23776 raft_consensus.cc:2410] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:15:25.159924 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.089232 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.013434 29337 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.182719 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.087050 29263 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:25.177799 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.118044 29275 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.119397 29334 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:25.184966 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.185385 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:25.188501 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:25.128993 29391 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:25.196679 23931 raft_consensus.cc:2393] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 14: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:25.118657 29375 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:25.084239 29274 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:25.078908 29053 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.213086 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:25.219465 25173 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:25.188198 29275 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:25.183575 29211 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:25.234093 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:25.085719 29299 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.235965 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.236972 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:25.188308 29334 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.248123 29334 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:25.249006 28999 consensus_meta.cc:287] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:15:25.249444 28999 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.249953 25173 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:25.252004 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:25.175889 29280 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.253187 23779 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
W20260709 12:15:25.253438 29405 log.cc:927] Time spent T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:15:25.254452 29280 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.210057 29391 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.256832 29337 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.257408 29391 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.257956 23784 raft_consensus.cc:3060] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:25.266222 29194 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:25.244304 28766 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.245093 28882 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.268352 28882 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.271270 28882 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.191190 29051 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.068943 23678 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 11 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.282209 29051 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.281893 23678 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.247505 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:25.284015 25168 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:25.285036 23783 raft_consensus.cc:2393] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 15: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:25.286444 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.231469 29390 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.240968 25169 raft_consensus.cc:606] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:25.289090 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.290215 29294 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.290580 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.290723 29294 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.291317 23855 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:25.279214 29280 consensus_queue.cc:1059] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:25.279883 29391 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.280395 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.287508 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.297765 23784 raft_consensus.cc:2468] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:25.293370 29391 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.310463 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.312341 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:25.304519 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.305753 23751 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:25.319351 23783 raft_consensus.cc:2393] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 13: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:25.200726 29263 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.321035 23855 raft_consensus.cc:2410] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:15:25.302691 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:25.323419 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.324021 25168 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:25.325338 23860 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:25.278266 29334 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.324138 23852 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:25.331880 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.333117 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:25.334103 29390 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:25.212671 29274 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.329326 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.342723 23783 raft_consensus.cc:2410] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:15:25.343138 23782 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:25.337430 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.326573 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.349859 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.350726 23852 raft_consensus.cc:2410] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:25.321527 29280 raft_consensus.cc:2764] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:15:25.362571 23777 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:25.315490 28621 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:25.340807 29390 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.368943 29334 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.369853 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:25.380434 23855 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.380039 23783 raft_consensus.cc:1240] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 16. Current term is 17. Ops: []
W20260709 12:15:25.395423 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49853us
W20260709 12:15:25.406936 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 157781us
W20260709 12:15:25.452272 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 225157us
I20260709 12:15:25.364805 29221 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:15:25.379712 29390 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:25.461690 25113 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.460505 29409 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:25.424610 29211 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.469821 29409 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.386824 29280 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.472286 29409 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.472411 29280 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:25.406615 29414 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:15:25.354362 29375 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.478180 23855 raft_consensus.cc:2468] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:15:25.408429 29408 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.472859 29409 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:25.523774 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 62172.9us
I20260709 12:15:25.525141 23782 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:25.525491 23749 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.545361 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.525278 29335 raft_consensus.cc:2749] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:25.561395 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.567646 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:25.582150 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.583369 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.587913 23927 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:25.578473 29299 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.601172 23825 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.564913 29344 raft_consensus.cc:3060] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:25.564168 29053 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.570297 29382 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:25.523017 29408 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.525094 29409 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.568441 29133 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.610153 23825 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.577312 29294 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:25.618144 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.617601 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:25.621482 23825 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.622100 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.620631 25169 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:25.578904 28999 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.649175 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:25.649848 25169 raft_consensus.cc:606] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:25.634706 23749 leader_election.cc:304] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.587080 29270 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:15:25.612514 29299 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:25.656106 29426 log.cc:927] Time spent T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:15:25.628105 29421 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.275s	user 0.001s	sys 0.001s
W20260709 12:15:25.656708 29419 log.cc:927] Time spent T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.247s	user 0.000s	sys 0.000s
I20260709 12:15:25.657433 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.662985 29409 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.663740 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:25.671229 29056 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.002s
I20260709 12:15:25.665424 29263 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.668218 29274 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.631395 28621 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:25.700613 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 65993.7us
I20260709 12:15:25.540148 23678 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:25.707764 23782 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:15:25.664327 23779 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:15:25.653152 29423 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:25.748306 29422 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 12:15:25.750126 29435 log.cc:927] Time spent T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:15:25.742880 23779 pending_rounds.cc:85] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 11
I20260709 12:15:25.751961 23779 pending_rounds.cc:107] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:15:25.752314 23779 raft_consensus.cc:2889] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:25.633109 29294 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.653589 29133 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:25.759276 29133 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.671053 29402 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:25.630856 29194 consensus_queue.cc:1048] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:15:25.765337 29280 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.717877 29391 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.767518 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:25.768201 23927 raft_consensus.cc:606] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:25.768528 23927 raft_consensus.cc:614] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:25.701004 29432 log.cc:927] Time spent T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:15:25.686652 29265 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:25.719312 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.707818 28886 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:25.712558 29417 log.cc:927] Time spent T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:15:25.762473 29408 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.709894 29205 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:15:25.712162 29056 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:25.744922 29420 log.cc:927] Time spent T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.243s	user 0.001s	sys 0.006s
I20260709 12:15:25.785466 23854 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:15:25.771749 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 163371us
I20260709 12:15:25.726619 29274 raft_consensus.cc:2749] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:25.779052 29431 log.cc:927] Time spent T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:15:25.726792 29409 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:15:25.800385 29133 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:25.775758 29304 consensus_meta.cc:287] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.006s
I20260709 12:15:25.801240 29304 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.798188 29344 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:25.804016 23929 raft_consensus.cc:2410] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:25.806063 29390 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.810597 29390 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.803372 29304 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.812008 23678 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 12, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.815629 23778 raft_consensus.cc:1275] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:25.814673 23678 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 9, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.790750 29275 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:25.840349 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 121122us
I20260709 12:15:25.841313 23749 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.797703 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:25.821259 25169 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:25.843747 25169 raft_consensus.cc:606] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:25.822069 29274 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:25.824816 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.822243 29334 raft_consensus.cc:993] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.828485 29396 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:15:25.845443 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47834.1us
I20260709 12:15:25.845615 23749 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.847095 29390 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:25.848088 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:25.853406 23749 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.848882 23778 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:25.850292 23899 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.855736 23899 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.856433 23780 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:25.855826 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:25.856963 29403 raft_consensus.cc:1081] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.861768 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.862908 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.846807 29430 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:25.865339 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:25.863538 23778 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:15:25.868022 29387 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:25.847823 29344 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:25.860862 29378 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.866183 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:25.860828 29304 raft_consensus.cc:2804] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:25.852658 29396 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:25.859745 29056 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.870551 23778 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:25.863338 29274 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:25.857174 29409 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:25.877490 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:25.855804 29277 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:15:25.881824 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:25.884225 25169 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:25.885494 29396 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.875751 29366 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:25.886627 29056 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:25.887058 29366 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:25.887508 29366 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:25.884258 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:25.898876 23778 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:25.900868 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.902113 25168 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: [13.14-15.15]
I20260709 12:15:25.876762 29304 raft_consensus.cc:697] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:25.876562 29378 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:25.903314 29304 consensus_queue.cc:237] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.910429 25113 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:25.907473 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:25.914304 29384 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:25.914773 29384 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:25.914917 29406 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:25.875125 29344 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.917091 29449 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:25.917716 29449 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.918886 29344 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.915220 29384 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.922183 25169 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:15:25.929315 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:25.932667 25170 raft_consensus.cc:2393] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:25.930596 29335 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:25.930783 29387 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:25.934141 29387 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:25.936180 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:25.937001 25168 raft_consensus.cc:2368] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13.
I20260709 12:15:25.937810 29387 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:25.938361 29449 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:25.938781 25119 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:25.880065 23678 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 12, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.939814 29136 raft_consensus.cc:2804] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:25.940361 29136 raft_consensus.cc:697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:25.940611 29275 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:25.943292 29334 consensus_queue.cc:1048] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:25.945932 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.946590 23930 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:25.947665 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.948380 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.949602 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.922868 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:25.951531 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.952378 23931 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:25.933178 29133 raft_consensus.cc:993] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.957258 29133 raft_consensus.cc:1081] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.940238 23678 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 11, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 11 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.961069 29379 raft_consensus.cc:993] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.961684 29379 raft_consensus.cc:1081] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:25.962219 23678 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 7 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:25.964845 29449 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:26.004245 29449 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.006433 29449 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:25.966028 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:25.969578 29136 consensus_queue.cc:237] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:25.973106 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:26.007790 23925 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:25.973834 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.947924 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:25.974314 23748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.974771 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:25.977006 29450 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
I20260709 12:15:25.977310 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.979914 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:25.983747 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:26.014619 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.015082 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:26.020025 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.987790 23898 leader_election.cc:400] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13.
I20260709 12:15:25.951474 23781 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:26.023017 23781 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.993992 23785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:25.994758 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed3fc49956b40aa86095410b7c7e5fa" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.995941 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.023952 23785 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:25.996659 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.996793 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:25.997061 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.024328 23785 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:25.997475 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.025130 23898 leader_election.cc:403] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:15:26.025831 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:25.998119 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.026965 23853 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 10.
I20260709 12:15:25.998302 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "349b687db5c44196bafc207c26ca3679"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:26.027961 23780 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:26.028314 23780 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.998347 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.029445 23857 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:26.029631 25170 raft_consensus.cc:1240] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:25.999361 29403 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:26.000250 29396 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:25.999373 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.002628 23931 raft_consensus.cc:1240] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:26.003064 29053 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:25.913642 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:26.012010 23932 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:26.030859 23854 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 16: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:26.053954 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.054874 29366 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.032748 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.013396 29449 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.057193 29449 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.034169 23860 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.057621 29449 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.033878 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.006474 29335 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.031535 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.059325 23856 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
W20260709 12:15:26.061344 23899 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.22.25.65:35817 timed out after 1.523s (SENT) [suppressed 10 similar messages]
I20260709 12:15:26.064530 29335 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.068307 29366 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.065938 29449 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.069072 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:26.070011 23929 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:26.071307 29449 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:26.006119 23678 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 11, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
W20260709 12:15:26.078627 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 91146.4us
I20260709 12:15:26.088210 29053 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:15:26.088423 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 174851us
I20260709 12:15:26.088819 29409 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.089813 23852 raft_consensus.cc:1240] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:26.089883 29409 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:26.090444 29409 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.095407 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:26.095839 29205 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.096355 29205 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.079737 23824 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:26.098487 29378 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:26.099040 29378 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:26.049543 23781 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.105682 29378 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.083737 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:26.085196 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.089237 23749 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:26.107415 23781 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.108220 29344 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:26.108603 29344 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.108946 29344 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.107512 23927 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:26.112550 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:26.112895 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:26.113379 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:26.113950 25173 raft_consensus.cc:3055] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:26.114255 28886 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.114609 29212 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:26.114972 28886 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.078725 29403 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.115437 28886 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.117023 29403 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.098881 29205 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.129565 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:26.033531 29318 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.133371 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:26.136711 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.137110 23851 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:15:26.082031 23776 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:26.104393 29335 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.114259 25173 raft_consensus.cc:740] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:26.089214 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.119892 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:26.123322 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.088733 29053 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:26.123718 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.155850 25114 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:26.079667 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:26.114773 23780 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.027489 28766 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.127789 23860 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:26.139961 29449 consensus_queue.cc:1059] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:15:26.154210 23782 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 18.
I20260709 12:15:26.166102 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.123510 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.087178 29056 raft_consensus.cc:3060] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:26.157927 23859 raft_consensus.cc:2368] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13.
I20260709 12:15:26.161645 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.169550 23860 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:15:26.170749 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 47336.7us
I20260709 12:15:26.172406 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.178110 23780 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.179977 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:26.180630 23927 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 18.
I20260709 12:15:26.182642 23749 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.183465 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.184363 23860 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:26.083280 29384 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.177254 23751 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:26.190557 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.191345 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.191817 23751 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:26.196535 23749 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.197001 23751 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.149617 29318 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.202095 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.209371 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:26.210315 25170 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:26.195811 29408 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:26.212096 23934 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.212294 23860 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:26.189358 29334 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:26.214828 23928 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:26.216001 29384 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:26.213806 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:26.217816 29308 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.175518 29442 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:26.194453 25173 consensus_queue.cc:260] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.219669 29441 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.213835 23678 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 10, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:26.220438 29263 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:26.195447 29274 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:15:26.221939 23751 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:26.222469 29100 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:26.215924 23852 raft_consensus.cc:1240] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:26.227130 25173 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:26.228292 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.223945 23900 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:26.240015 23900 leader_election.cc:400] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13.
I20260709 12:15:26.193763 29265 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:26.200193 29270 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:15:26.224524 29450 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:15:26.245213 23751 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:26.253597 23855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.255476 23855 raft_consensus.cc:2368] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14.
I20260709 12:15:26.194311 29391 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:26.229882 29275 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:26.211812 29318 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:26.199507 29379 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.244508 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:26.224380 29280 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.284603 25173 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:26.287688 23934 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 18.
I20260709 12:15:26.240932 29449 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:26.195158 29402 consensus_queue.cc:1059] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 12:15:26.326216 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 81795us
I20260709 12:15:26.329165 23781 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:26.331243 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.295374 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.332482 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.336184 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:26.336555 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.336800 23781 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.230254 29373 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:26.254091 29270 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.251518 29396 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:26.251479 29334 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:26.288307 29441 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:26.283914 29299 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.344009 23859 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 18: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:26.243892 29384 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:15:26.240172 29409 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:26.298612 29416 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:15:26.300206 29280 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.351910 29270 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:26.340219 29373 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.370635 29280 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.370955 29373 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:26.367178 23781 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:26.298026 29275 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:26.342689 29051 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:26.372994 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:26.373441 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 129884us
I20260709 12:15:26.474256 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:26.371866 29423 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:26.369609 29441 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:26.506029 29408 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:26.509544 29451 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:15:26.511888 29335 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.550796 29344 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.557250 29409 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:26.566849 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 267735us
I20260709 12:15:26.585000 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
W20260709 12:15:26.460855 29457 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:15:26.369021 29299 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.649427 23777 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 12:15:26.542810 28886 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.515266 23785 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.606592 29408 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.660597 23783 raft_consensus.cc:1240] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:15:26.656836 29334 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.666097 29299 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.702462 29408 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.703613 23776 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:26.704348 23856 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 16: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:26.712198 23785 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.715528 29441 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:26.734001 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 79300.6us
I20260709 12:15:26.713131 29344 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.658761 29275 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.699093 29270 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.740887 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:26.738888 29475 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:15:26.747931 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:26.757225 23782 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:26.759147 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:26.759982 23784 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:26.739399 29280 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:26.786609 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.788398 29335 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:26.789016 29335 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.791478 29335 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:26.794857 29304 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.795456 29304 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.797734 29304 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:26.802155 23900 leader_election.cc:400] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14.
I20260709 12:15:26.802563 23900 leader_election.cc:403] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:26.802640 29467 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:26.803136 29467 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:26.804046 29467 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.806020 29449 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:26.806131 29384 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.807085 29449 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.807453 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:26.809055 23931 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.805342 29467 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:26.816125 29384 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 13. Current term is 14.
I20260709 12:15:26.817757 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:26.818470 23777 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:26.820242 23778 raft_consensus.cc:1240] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:15:26.821808 29211 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:26.822516 29211 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.823853 29280 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:26.825126 29056 consensus_meta.cc:287] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.001s
I20260709 12:15:26.825543 29056 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13.
I20260709 12:15:26.827207 29211 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:26.841225 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44692.2us
I20260709 12:15:26.844059 23931 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:26.847872 23851 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:26.876992 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:26.886976 23899 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:26.888821 23899 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:26.825448 29318 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:26.854972 28886 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:26.898765 29056 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:26.859459 29373 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:26.899714 29211 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:15:26.943173 29449 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:26.926380 29318 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.966111 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:26.985683 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:26.986626 25168 raft_consensus.cc:3055] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:26.987097 25168 raft_consensus.cc:740] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:26.989653 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:26.988741 25168 consensus_queue.cc:260] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:26.991334 25168 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:26.999132 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 99065.1us
I20260709 12:15:27.001240 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.001982 23927 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:27.006006 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:27.006904 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:27.007694 25175 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:27.008054 25175 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
W20260709 12:15:26.995182 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 106458us
I20260709 12:15:27.015693 23779 raft_consensus.cc:1240] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:27.021068 29053 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.007148 29270 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:27.022845 29053 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:27.008869 25175 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.027694 25168 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:27.028276 25175 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:27.037230 23934 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45620 (request call id 3151) took 1021 ms. Trace:
I20260709 12:15:27.037742 23934 rpcz_store.cc:276] 0709 12:15:26.015460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.016964 (+  1504us) service_pool.cc:224] Handling call
0709 12:15:27.036976 (+1020012us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b54000f6ed4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:27.037145 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121952000}
I20260709 12:15:27.040194 23928 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:15:27.045190 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:27.045831 23780 raft_consensus.cc:1240] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:27.060410 25175 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:27.061064 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:27.062281 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:27.062966 23779 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:27.063930 23933 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:27.068961 23749 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:27.072572 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.073652 23932 raft_consensus.cc:2393] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:27.046003 23929 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:27.079658 23927 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:27.081866 23930 raft_consensus.cc:1240] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:15:27.083204 23749 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:27.084801 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.085441 25168 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
W20260709 12:15:27.086182 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 71909.6us
I20260709 12:15:27.086297 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.086836 23927 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:27.086961 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.092214 29275 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:27.092832 29275 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:27.093757 29275 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.117422 29475 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.118075 29475 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.120143 25170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44500 (request call id 775) took 1040 ms. Trace:
I20260709 12:15:27.120548 25170 rpcz_store.cc:276] 0709 12:15:26.080190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.080934 (+   744us) service_pool.cc:224] Handling call
0709 12:15:26.081243 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.210742 (+129499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.119905 (+909163us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54008633d4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:27.120077 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296136576}
I20260709 12:15:27.122224 29437 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:27.122668 29437 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:27.122749 23927 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:27.123553 29437 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.123915 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:27.124919 29437 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:27.132170 29472 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:27.134617 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47746.2us
I20260709 12:15:27.136058 23899 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:27.144922 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:27.145711 23930 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:27.034957 29449 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:15:27.132948 29472 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.162886 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.163853 23899 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.133561 29390 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:27.177596 23856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56656 (request call id 3638) took 1004 ms. Trace:
I20260709 12:15:27.144724 29484 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:27.153173 29269 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:27.184356 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.178112 23856 rpcz_store.cc:276] 0709 12:15:26.160529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.160868 (+   339us) service_pool.cc:224] Handling call
0709 12:15:27.164970 (+1004102us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54003c0854. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:27.165136 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220956544}
I20260709 12:15:27.186545 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:27.160415 29373 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:27.169499 29430 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:27.157148 29375 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:27.192798 29390 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:27.190272 29318 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.193231 29387 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:27.210914 29390 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.203899 29308 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:27.230517 29430 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:27.244491 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 118817us
I20260709 12:15:27.246168 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:27.227963 29471 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:27.251538 29475 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.235267 29308 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.250459 23824 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:27.225904 29487 consensus_queue.cc:1059] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:15:27.250936 29468 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:27.259549 29308 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:27.273494 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:27.274436 23852 raft_consensus.cc:2393] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 16: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:15:27.275317 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:27.281261 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.261039 29471 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.270893 29468 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.282356 29468 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:27.283699 29491 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:15:27.284060 23779 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 19.
I20260709 12:15:27.284569 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:27.262504 29373 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:27.291165 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:27.291679 29472 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.295249 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:27.299402 23854 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:27.299894 29430 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.281302 29434 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:27.307389 29434 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:27.304075 25169 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:27.282837 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.309142 23781 raft_consensus.cc:2410] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:15:27.310983 23852 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:27.315539 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:27.329588 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 125158us
I20260709 12:15:27.310165 29277 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:27.344245 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.345978 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.344275 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:27.350682 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 149981us
I20260709 12:15:27.327633 29056 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:15:27.341161 29408 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:27.349161 29269 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:27.350723 29500 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:27.359215 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.336110 29053 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:15:27.350069 29434 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.366564 29484 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:27.367422 29434 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.369300 23783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 3080) took 1246 ms. Trace:
I20260709 12:15:27.369704 23783 rpcz_store.cc:276] 0709 12:15:26.122931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.123004 (+    73us) service_pool.cc:224] Handling call
0709 12:15:26.123283 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.690973 (+567690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.369010 (+678037us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b540041c654. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b2e01 00007fb2ad7b183e 00007fb2b2776419 00007fb2abcedbd4 00007fb2abcedb62 00007fb2abceda91 00007fb2abceda0c 00007fb2abceccc2 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:27.369205 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235106048}
I20260709 12:15:27.363057 29056 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:15:27.372401 29056 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:27.372761 29056 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:27.383956 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.384653 23927 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:27.389386 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.364498 29500 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.399755 23927 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 19.
I20260709 12:15:27.403419 29500 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:27.409488 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:27.413987 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.429504 29053 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.443612 29053 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.447492 29053 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:27.449739 28999 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:27.362653 29468 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.435317 23824 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:27.459362 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.466784 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:27.469877 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.373667 29056 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.475214 25168 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 12:15:27.459684 29269 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.478047 25169 raft_consensus.cc:2410] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 12:15:27.410075 29402 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:15:27.489480 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:27.465620 29497 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:27.460719 29501 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:27.501703 29497 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:27.504635 29497 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.512238 23852 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:27.500000 28886 consensus_queue.cc:1059] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:15:27.514473 28886 consensus_queue.cc:1243] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:15:27.518465 23781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43438 (request call id 3094) took 1122 ms. Trace:
I20260709 12:15:27.518980 23781 rpcz_store.cc:276] 0709 12:15:26.396400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.584636 (+188236us) service_pool.cc:224] Handling call
0709 12:15:27.518212 (+933576us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54001eebd4. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71b571 00007fb2ad7b62fa 00007fb2b2777b17 00007fb2abcf26d4 00007fb2abcf2662 00007fb2abcf2591 00007fb2abcf250c 00007fb2abcf17c2 00007fb2abaed48c 00007fb2abaec8c6 00007fb2abaec26f
0709 12:15:27.518383 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590000896}
I20260709 12:15:27.491930 29475 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:15:27.528909 29408 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:15:27.534133 29458 log.cc:927] Time spent T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 12:15:27.541355 29471 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.502892 29308 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.545382 23927 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:27.538471 29373 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:27.547081 29373 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:27.548138 29373 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.549849 29373 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.499260 29501 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.554584 23779 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:27.557003 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:27.557243 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:27.557739 23928 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 18.
I20260709 12:15:27.559957 29308 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.560990 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:27.561687 29468 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.565116 28766 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:27.575996 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.579974 23749 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:27.580605 29434 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:27.581048 29280 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:27.581699 29280 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:27.582623 29280 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.585057 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:27.560971 23930 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:27.595171 29056 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:27.604163 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:27.611552 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:27.612489 25168 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:27.614322 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:27.614966 23928 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:27.634078 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:27.634821 25170 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:27.605569 28999 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:27.606076 29499 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:27.652271 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:27.613706 29136 consensus_queue.cc:1059] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:15:27.617444 29468 consensus_queue.cc:1059] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:15:27.649700 28886 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:27.714723 23898 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.670406 23784 raft_consensus.cc:1275] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:27.680596 28621 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:27.711843 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 221679us
I20260709 12:15:27.760887 23928 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:27.771420 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:27.658814 28999 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.745661 29379 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:27.660059 29499 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.761785 29363 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:27.809547 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.810379 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44907.3us
I20260709 12:15:27.742223 28886 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.779218 29053 raft_consensus.cc:493] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.780663 29379 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:27.827762 23783 raft_consensus.cc:1240] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:15:27.888581 25168 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:27.814991 29363 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.840058 29391 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:27.911724 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:27.885963 29275 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:27.825466 29053 raft_consensus.cc:515] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:27.937942 23751 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.940793 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
W20260709 12:15:27.947460 29517 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:15:27.953526 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44368.2us
I20260709 12:15:27.960294 25170 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:27.967724 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:27.977422 29520 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:15:27.977978 29519 log.cc:927] Time spent T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:15:27.980769 29363 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:27.986719 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:27.924299 29391 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.929688 29275 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:27.930645 29379 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:27.989837 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.031926 29408 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:27.987867 29280 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:27.985455 29501 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:28.010602 29521 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
W20260709 12:15:28.006925 29518 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.203s	user 0.000s	sys 0.003s
I20260709 12:15:27.991822 29390 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:27.993330 28999 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:27.999377 29053 leader_election.cc:290] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.034845 29379 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:28.035735 29280 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:28.049906 29280 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:28.034255 29499 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.059774 29280 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:28.060320 29373 consensus_meta.cc:287] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.007s
I20260709 12:15:28.074550 29467 raft_consensus.cc:3055] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:28.075091 29467 raft_consensus.cc:740] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:28.075959 29467 consensus_queue.cc:260] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.077258 29467 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:28.082618 29525 log.cc:927] Time spent T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 12:15:28.083680 29280 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.086994 29275 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:27.989508 29531 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:15:28.105532 28886 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:28.124785 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.130532 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:28.130920 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.132568 29136 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:28.134042 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:28.135221 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.135810 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.136444 23857 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:15:28.135838 29391 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:28.107035 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 75904us
I20260709 12:15:28.140934 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:28.161062 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.161909 23927 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:28.162907 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.167971 23928 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:28.180749 23824 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:28.181398 25119 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:28.187656 23930 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:15:28.190521 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49687us
I20260709 12:15:28.203558 23930 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:28.204723 23926 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:15:28.206363 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 217969us
I20260709 12:15:28.202332 23825 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.184677 28999 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:28.208839 23899 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.216806 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:28.213732 23859 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:28.200322 29280 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:28.219388 23825 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.210110 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:28.202584 29515 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:28.239950 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.208463 28766 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.249622 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.222261 29475 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:28.261780 25175 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:15:28.270234 28766 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.231861 28999 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:28.280586 28999 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.281499 28766 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.257326 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:28.257088 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:15:28.284157 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44302.4us
I20260709 12:15:28.241101 29275 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.224995 29382 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:28.224872 29280 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.241830 29515 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.242408 29205 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:28.287297 29430 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:28.220237 29391 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:15:28.212705 29194 consensus_queue.cc:1048] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:15:28.252704 29487 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:28.295231 25168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:28.304440 25168 raft_consensus.cc:606] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.293989 29280 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:28.255658 29497 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:15:28.294250 29205 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:28.308660 29205 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:28.254634 29471 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:15:28.316030 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5bb29269e8a4aa1b79f91e4ed687053" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.317538 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:28.318070 29308 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:28.318620 29308 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.295272 29275 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.303201 29531 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:28.321203 29390 raft_consensus.cc:493] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:28.321781 29390 raft_consensus.cc:515] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.321488 29275 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.324151 29308 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:28.326678 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.328469 29472 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:28.331489 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:28.328079 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 108770us
I20260709 12:15:28.293670 29515 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:28.352489 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:28.354023 29391 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:28.355290 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.356112 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.356905 23785 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:15:28.357275 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.356228 23783 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:28.355460 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.359112 23777 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
W20260709 12:15:28.362004 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 92801.6us
I20260709 12:15:28.371223 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.372032 23852 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:28.377518 29053 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:28.377948 23749 leader_election.cc:304] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.383353 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.384215 23927 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:28.385121 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.386329 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.391528 23749 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.394198 25175 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:28.407363 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:28.408842 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 53733.4us
I20260709 12:15:28.410342 23749 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.412343 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.413319 23927 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:28.377194 29545 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:15:28.418159 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.427066 25170 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:28.435281 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:28.404475 29390 leader_election.cc:290] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.436008 25170 raft_consensus.cc:606] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.438346 23825 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.438973 25170 raft_consensus.cc:614] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:28.415716 29512 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:28.428285 29275 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:28.417255 29406 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:28.447180 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:28.429176 29499 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:28.464627 25113 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:28.465899 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 213417us
I20260709 12:15:28.469223 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.469749 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.472108 23751 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.437777 29505 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.440304 29387 raft_consensus.cc:493] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:28.455099 29390 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:28.499676 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 52596.4us
I20260709 12:15:28.453367 29497 raft_consensus.cc:2749] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.493662 29275 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:15:28.519650 29408 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.453557 29533 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 12:15:28.564769 29537 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.281s	user 0.005s	sys 0.000s
I20260709 12:15:28.460192 29512 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.511802 29387 raft_consensus.cc:515] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.557974 29484 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:28.536453 29532 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:28.476271 29406 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.478628 29053 raft_consensus.cc:2749] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:28.566267 25175 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
W20260709 12:15:28.573770 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 138149us
I20260709 12:15:28.575639 25175 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.475446 29467 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:15:28.573099 29362 raft_consensus.cc:993] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:28.550531 29390 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.590253 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3eeb67716b0e4f49aa1a4bd5815f684b" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.577153 29532 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:28.615967 29561 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:15:28.588783 29547 raft_consensus.cc:2749] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:28.621676 29406 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.588615 29390 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.577571 29205 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.579164 29391 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:15:28.629578 29205 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.630570 23854 raft_consensus.cc:2410] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:28.592658 29133 raft_consensus.cc:1081] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:28.634145 23783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:28.634829 23783 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:28.635200 23783 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.601747 29532 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:28.646588 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.647460 23927 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:28.648759 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:28.650924 23824 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:28.669088 23783 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.577991 29484 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:28.691658 29554 log.cc:927] Time spent T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:15:28.693367 29390 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.702695 29468 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:28.736104 29468 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.740918 29468 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.748353 29280 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:28.750438 29548 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.750906 29280 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.751863 23780 raft_consensus.cc:1240] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:28.752221 29512 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:28.755474 29548 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.820438 29548 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.758800 29387 leader_election.cc:290] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:28.760624 29515 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.823732 29515 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:28.824612 25168 raft_consensus.cc:2368] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 17. Current term is 18.
I20260709 12:15:28.703420 29484 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:28.703693 29205 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:15:28.772698 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65375.3us
I20260709 12:15:28.825711 29205 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:28.826211 29205 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:28.780040 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:28.781462 29532 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:28.801816 29559 log.cc:927] Time spent T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 12:15:28.831862 29532 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:28.833953 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.834661 23925 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:28.836299 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.837613 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:28.852006 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:28.872691 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:28.875517 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:28.876245 23933 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:28.880156 23898 leader_election.cc:400] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Vote denied by peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683) with higher term. Message: Invalid argument: T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 17. Current term is 18.
I20260709 12:15:28.889860 23783 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:28.855922 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.862138 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:28.904544 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:28.908164 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:28.909997 29546 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:28.895364 23932 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:28.922616 23859 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:28.924081 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.927018 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:28.929751 23857 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.930965 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:28.854965 23781 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:28.938760 23857 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:28.945024 23934 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:15:28.952577 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 65979.6us
I20260709 12:15:28.958268 23934 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:28.959044 23934 raft_consensus.cc:606] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:28.962594 25173 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:28.964879 29541 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:28.966444 29548 consensus_queue.cc:1059] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:28.967701 29548 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:28.968153 29548 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:28.969172 29548 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.970504 29548 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.979573 29515 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:28.980289 29546 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:28.965654 29541 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:28.984613 29541 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:28.984763 23779 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:28.985826 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.986021 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:28.986670 23781 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:28.986778 23780 raft_consensus.cc:2393] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:28.988700 23859 raft_consensus.cc:1240] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 05679ef5e2b74861aad2a08b6a05e276 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:28.988993 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:28.989734 23780 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:28.990027 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:28.989734 23933 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:15:28.990716 23857 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:15:28.991622 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:28.992203 23779 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:28.992607 23779 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:28.992129 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:29.001353 23900 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:29.002105 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 71723.7us
I20260709 12:15:29.008320 23857 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:15:29.009447 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.004961 23751 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:28.961151 29205 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.018260 29205 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:29.021890 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:29.023371 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.024327 23781 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 18: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:29.025080 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956a6729b1f34c499670aa77b5896a53" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.035856 29437 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.019726 29562 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:29.036530 29437 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.040937 25173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.042282 25173 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:15:29.036207 29515 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:29.044987 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.047515 29562 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:29.048486 29562 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.051872 29515 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:29.052886 29515 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.054255 29484 raft_consensus.cc:2764] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:15:29.026392 29546 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:29.039556 29437 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:29.055531 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 44266.6us
I20260709 12:15:29.066092 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:29.066990 23851 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:15:29.055032 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:29.056559 29546 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:29.064381 23779 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.074033 23931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:29.074681 23931 raft_consensus.cc:606] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:29.075076 23931 raft_consensus.cc:614] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:29.073927 29468 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:29.075965 29468 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:29.076376 29468 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.076066 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.078888 23779 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.074579 23749 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.080889 23749 leader_election.cc:304] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.082340 29275 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:29.083689 29536 raft_consensus.cc:2749] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:29.069828 29546 raft_consensus.cc:3060] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:29.093276 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:29.095861 23900 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:29.100308 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.103293 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:29.105556 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:29.107024 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:29.092253 29548 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:29.092094 29536 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:29.119524 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:29.124511 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:29.127429 29540 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.128050 29540 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.103945 29308 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:29.103695 29505 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:29.131072 29505 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.143525 25170 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:29.152355 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:15:29.160349 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 48155.2us
I20260709 12:15:29.160552 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.161150 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:29.155038 25170 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:29.164556 25170 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:29.166610 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:29.167495 23783 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 11: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:29.167783 23899 leader_election.cc:304] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:29.169077 23899 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:29.106097 29467 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:15:29.179754 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:29.180630 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:29.181258 23854 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:29.162479 28999 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.178910 25170 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.186300 28999 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.187294 25170 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:29.187937 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:29.189466 29501 consensus_queue.cc:1059] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:29.160331 29406 consensus_queue.cc:1059] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:15:29.196167 23851 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:15:29.196794 23931 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:29.197266 29513 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:29.198628 29205 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:29.199667 29390 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:29.155076 29536 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.204754 23779 raft_consensus.cc:2410] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:15:29.205389 23854 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:29.209456 28999 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.211738 29562 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:29.213064 29406 consensus_queue.cc:1243] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:29.213708 29540 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:29.214978 25170 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 12.
I20260709 12:15:29.217062 29533 raft_consensus.cc:993] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:29.217523 29533 raft_consensus.cc:1081] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:15:29.199499 29469 consensus_queue.cc:1175] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "438159d8de3c4027bdcf57eb81dedf86" responder_term: 11 status { last_received { term: 11 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 11 index: 10 } }
I20260709 12:15:29.140138 29548 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.183389 29512 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:29.209372 29505 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.213615 29406 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:29.225183 29406 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.226492 29406 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.211167 29532 raft_consensus.cc:2749] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:29.230962 29548 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:29.233510 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 53437.1us
I20260709 12:15:29.200201 29390 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.199481 29468 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.253334 29468 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:29.222846 23749 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.232916 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.257555 23749 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:29.259472 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56878.4us
I20260709 12:15:29.262199 23824 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:29.264379 29536 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.269220 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:29.266777 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:29.270080 23851 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:29.274915 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.275707 25172 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:29.277956 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:29.280089 29501 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:29.280710 29501 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.281945 29501 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.221089 29546 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.291177 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:29.292160 23779 raft_consensus.cc:2393] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 18: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:29.293857 29546 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:29.271150 29390 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.302446 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.303823 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.311407 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:15:29.316784 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59311.5us
I20260709 12:15:29.313072 23926 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:15:29.349067 25113 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.346674 23934 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:29.363677 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:29.270119 29378 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:29.328604 29545 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:29.392948 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.393478 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:29.394073 23779 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:29.393803 25172 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 16: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:29.394228 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:29.395920 23783 raft_consensus.cc:2393] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 16: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:29.395944 23782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:15:29.396776 23782 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.397118 23782 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.397392 23825 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.399223 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.400013 23934 raft_consensus.cc:2393] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 16: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:29.402853 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.406024 28621 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:29.406458 28621 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.406754 28621 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:29.410933 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.411731 25170 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:29.414139 23925 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 17: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:15:29.404736 29378 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.421605 29391 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:29.421368 29378 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.422381 29391 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.425580 29391 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.386514 29497 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:29.426266 29497 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.387634 29536 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:29.405758 29275 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:29.387077 29390 raft_consensus.cc:493] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.428033 29390 raft_consensus.cc:515] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.387908 29362 raft_consensus.cc:2749] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:29.439378 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 265296us
I20260709 12:15:29.432011 29390 leader_election.cc:290] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.452491 25119 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:29.458608 28621 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.495846 28621 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:29.461568 23782 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.497911 23782 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:29.455618 29362 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:29.435561 29275 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:29.506364 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 51998.9us
W20260709 12:15:29.522585 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 116855us
W20260709 12:15:29.522161 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 118175us
W20260709 12:15:29.511283 29567 log.cc:927] Time spent T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:15:29.530970 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.534438 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
W20260709 12:15:29.534714 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 184157us
I20260709 12:15:29.546993 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:29.548029 25169 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:29.553408 23859 raft_consensus.cc:2468] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:15:29.558595 25119 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:29.541862 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:29.569196 23781 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:15:29.569234 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 291344us
I20260709 12:15:29.591661 29362 raft_consensus.cc:2804] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:29.609935 23859 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:29.607844 23781 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:29.622715 23859 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:15:29.634919 29572 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:15:29.606177 29362 raft_consensus.cc:697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:29.641383 29362 consensus_queue.cc:237] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.645315 29548 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:29.554939 29497 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.649737 25169 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 12:15:29.603662 29275 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.665132 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:29.665864 25170 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:29.670229 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.673002 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:29.673808 23857 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.681668 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:29.682331 23928 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:29.684068 23749 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:29.689136 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f333e0a484248ce88f0905e18500838" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:29.690117 23859 raft_consensus.cc:2410] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:29.690867 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.691619 23934 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:29.691809 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:29.692370 23933 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:29.692658 23933 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:29.698030 23857 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:29.700367 23859 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
W20260709 12:15:29.703760 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 296043us
I20260709 12:15:29.693387 23933 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.702155 29408 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:29.700775 29583 raft_consensus.cc:2749] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:29.716486 28621 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:15:29.715102 23933 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.705521 25170 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:29.709520 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:29.737357 23928 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
W20260709 12:15:29.740808 29574 log.cc:927] Time spent T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
W20260709 12:15:29.760191 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 81828.9us
I20260709 12:15:29.779197 23933 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:29.732277 29408 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
W20260709 12:15:29.791908 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 160570us
W20260709 12:15:29.792660 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.713737 29592 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:15:29.793208 29408 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:29.819769 29406 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:29.831300 29579 log.cc:927] Time spent T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:15:29.837445 23751 leader_election.cc:304] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:29.841216 23751 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:29.842103 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 119178us
W20260709 12:15:29.843662 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 115350us
W20260709 12:15:29.846751 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.849087 23928 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:29.840358 29390 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.853500 29275 raft_consensus.cc:2749] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:29.861451 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 154446us
I20260709 12:15:29.865602 29586 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:29.856986 29390 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.876014 23824 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:29.880401 29472 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:29.881009 29472 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:29.857232 29592 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:15:29.882004 29472 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:29.869040 29586 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:29.887712 29586 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:29.893149 29390 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:29.902474 23782 raft_consensus.cc:1240] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:29.917994 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:29.928522 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 80307.7us
I20260709 12:15:29.941987 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:29.947487 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 74098.6us
I20260709 12:15:29.949321 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:29.950570 23928 raft_consensus.cc:2410] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:15:29.939674 29390 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:15:29.950649 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 84931us
W20260709 12:15:30.000335 29605 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:15:30.010208 29408 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:30.014681 29596 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:15:30.015502 29501 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:29.953635 29546 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 12:15:30.021421 29580 log.cc:927] Time spent T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.441s	user 0.003s	sys 0.000s
I20260709 12:15:30.050812 23783 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:30.052780 29136 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:30.086323 23928 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:15:30.090322 29570 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.510s	user 0.003s	sys 0.000s
I20260709 12:15:30.097718 29408 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:30.139909 29584 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:30.147413 23851 raft_consensus.cc:1275] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:30.151567 29136 consensus_queue.cc:1048] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:30.131222 23682 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 9 to 10, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 10 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:15:30.187378 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 226889us
I20260709 12:15:30.244340 23781 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:30.261252 23928 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:30.261884 23785 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:30.267885 23785 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:30.267568 29604 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:15:30.352931 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:30.356849 25172 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:30.373452 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:30.374331 23932 raft_consensus.cc:2393] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ee4ab0747914e7abc642312517b5e25 in current term 17: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:15:30.406452 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 46046.4us
I20260709 12:15:30.414021 23927 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:15:30.377223 29618 log.cc:927] Time spent T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.059s	user 0.003s	sys 0.000s
I20260709 12:15:30.426703 25168 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:30.431057 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:30.433622 29471 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.435808 29299 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:30.436004 29471 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:30.438006 29471 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:30.440442 29467 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:30.453249 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:30.465687 23854 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:30.469049 23851 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:30.472314 29534 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.472919 29534 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:30.477685 29606 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.478329 29606 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:30.480564 29606 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:30.481190 29585 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:30.485185 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:30.496687 29586 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:30.427886 29584 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:30.430145 29545 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:15:30.503166 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:30.504593 25168 raft_consensus.cc:2410] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:15:30.506492 23934 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:30.510229 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:30.475438 29534 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:30.514979 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:30.519054 23782 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 19.
I20260709 12:15:30.463465 29471 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:15:30.526273 23783 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:30.522898 29275 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:15:30.539300 29583 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.548436 29583 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:30.561630 29583 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:30.533697 23749 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:30.595563 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 61812us
I20260709 12:15:30.612049 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:30.624208 29391 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:30.625279 29391 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:30.656330 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:30.683311 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:30.684182 23851 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:30.686828 25168 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:30.688685 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:30.627079 29391 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:30.700433 29626 log.cc:927] Time spent T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.252s	user 0.000s	sys 0.003s
W20260709 12:15:30.703802 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 92289.1us
I20260709 12:15:30.707373 29378 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:30.708063 29378 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.708452 29378 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:30.716928 29501 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:15:30.740774 29624 log.cc:927] Time spent T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
I20260709 12:15:30.764961 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:30.767612 29275 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:30.768225 29275 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.768697 29275 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:30.774824 29387 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:30.801735 29387 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:30.806881 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:30.822499 29545 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:30.837836 29499 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.838812 29499 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:30.839676 29387 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:30.841892 29499 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:30.849134 29499 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:15:30.851507 29499 consensus_queue.cc:1243] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 05679ef5e2b74861aad2a08b6a05e276 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:15:30.872121 29469 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.872920 29469 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:30.875895 29469 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:30.895788 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:30.896677 23859 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:30.898674 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:30.899745 29469 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:30.900174 29469 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.900548 29469 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:30.915341 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:30.887925 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:30.921365 29469 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:30.923961 29408 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:30.936349 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:30.974985 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:30.973354 29378 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:30.978273 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:30.990482 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:30.998685 23857 raft_consensus.cc:2368] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 19. Current term is 20.
I20260709 12:15:30.979099 29469 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:31.006045 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:31.016991 25168 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:15:31.022993 29378 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 20 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:31.027725 29534 raft_consensus.cc:2749] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:31.028887 25172 pending_rounds.cc:85] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Aborting all ops after (but not including) 11
I20260709 12:15:31.029230 25172 pending_rounds.cc:107] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:15:31.029522 25172 raft_consensus.cc:2889] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:31.025684 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:31.026350 29469 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:31.059859 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 69732.7us
I20260709 12:15:31.012511 29275 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:31.076807 25170 raft_consensus.cc:2393] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 13: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:15:31.081693 23900 leader_election.cc:400] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 19. Current term is 20.
I20260709 12:15:31.082115 23900 leader_election.cc:403] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:31.083878 29512 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:31.095553 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:31.125242 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e60a1c6520545729d27474d8f237ce5 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:50860
W20260709 12:15:31.129398 23668 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet de3c70defd864769b26b52b7f36548cd with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 6
W20260709 12:15:31.157218 29636 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:31.177052 29275 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:31.184235 23678 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 275) took 28030 ms (client timeout 15000 ms). Trace:
I20260709 12:15:31.194643 29391 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:31.195819 29391 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:31.196213 29391 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:31.197306 29391 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:31.198625 29391 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.206712 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:31.230154 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:31.233428 23780 raft_consensus.cc:2393] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 19: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:31.233548 25174 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:15:31.184788 23678 rpcz_store.cc:269] 0709 12:15:03.153911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.154180 (+   269us) service_pool.cc:224] Handling call
0709 12:15:31.183984 (+28029804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:27.033880 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:27.128703 (+ 94823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:27.128756 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:15:27.128776 (+    20us) tablet.cc:662] Decoding operations
0709 12:15:27.130380 (+  1604us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:27.130495 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:27.130526 (+    31us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:15:27.210695 (+ 80169us) tablet.cc:701] Row locks acquired
0709 12:15:27.210750 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:15:27.210928 (+   178us) write_op.cc:270] Start()
0709 12:15:27.211096 (+   168us) write_op.cc:276] Timestamp: P: 1783599327210863 usec, L: 0
0709 12:15:27.211121 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:15:27.261442 (+ 50321us) log.cc:844] Serialized 7080 byte log entry
0709 12:15:27.545683 (+284241us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa5ebc85 00007fb2aa5ec6a2 00007fb2ad35e1db 00007fb2ad359aaf 00007fb2aa6a80ac 00007fb2b35029a9 00007fb2aa6c469b 00007fb2aa6c293f 00007fb2aa6c4c02 00007fb2ad34c20b
0709 12:15:28.304040 (+758357us) op_driver.cc:464] REPLICATION: finished
0709 12:15:28.369159 (+ 65119us) write_op.cc:301] APPLY: starting
0709 12:15:28.369248 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:28.768498 (+399250us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:28.768526 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:29.086337 (+317811us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:29.115769 (+ 29432us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:15:29.115820 (+    51us) write_op.cc:312] APPLY: finished
0709 12:15:29.116232 (+   412us) log.cc:844] Serialized 164 byte log entry
0709 12:15:29.116408 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:29.116592 (+   184us) write_op.cc:454] Released schema lock
0709 12:15:29.116665 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:29.117037 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:29.117070 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":136948,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":64825,"apply.run_cpu_time_us":27500,"apply.run_wall_time_us":747870,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":191176,"prepare.run_cpu_time_us":32380,"prepare.run_wall_time_us":710284,"raft.queue_time_us":156047,"raft.run_cpu_time_us":19512,"raft.run_wall_time_us":124791,"replication_time_us":1074082,"spinlock_wait_cycles":58313472,"thread_start_us":765178,"threads_started":4,"wal-append.queue_time_us":523719,"wal-append.run_cpu_time_us":23014,"wal-append.run_wall_time_us":2313482}]]}
I20260709 12:15:31.246251 29512 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 19. Current term is 20.
I20260709 12:15:31.265200 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:31.269060 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:31.276578 23859 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.296247 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:31.299158 23931 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 20.
W20260709 12:15:31.310086 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 48778.2us
W20260709 12:15:31.340492 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 69712.8us
W20260709 12:15:31.341840 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 80241.7us
I20260709 12:15:31.364212 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:31.364948 23777 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:31.352161 28886 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.379489 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 4e60a1c6520545729d27474d8f237ce5 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:31.391083 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:15:31.498659 29640 log.cc:927] Time spent T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:15:31.522697 29513 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:31.523398 29513 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:31.524641 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 120348us
I20260709 12:15:31.524825 28886 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:31.525731 29583 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:31.565481 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 159536us
I20260709 12:15:31.570487 23777 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 20.
I20260709 12:15:31.588886 23859 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:31.607532 23932 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:31.608893 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:31.612329 23785 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15.
I20260709 12:15:31.616245 29648 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:31.636533 29648 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:31.664134 29648 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:31.707886 29513 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:31.739980 23899 leader_election.cc:400] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15.
I20260709 12:15:31.740403 23899 leader_election.cc:403] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:15:31.758908 29664 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:15:31.769568 29643 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.775527 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:31.776394 23854 raft_consensus.cc:2368] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15.
I20260709 12:15:31.765590 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:31.796154 23900 leader_election.cc:400] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Vote denied by peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15.
I20260709 12:15:31.822161 29654 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:31.830279 29654 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:31.849889 29643 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) with higher term. Message: Invalid argument: T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da for earlier term 14. Current term is 15.
I20260709 12:15:31.835325 29654 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:31.948076 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46583.3us
W20260709 12:15:32.000622 29604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.026s	sys 0.024s
W20260709 12:15:32.001123 29604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.026s	sys 0.024s
I20260709 12:15:32.017951 29607 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:32.018724 29607 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.031786 29536 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:15:32.043802 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:32.056203 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:32.046604 29607 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:32.080366 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:32.098767 29582 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:32.152374 23785 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:32.172569 28621 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:15:32.171646 23677 catalog_manager.cc:5697] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 14, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.173228 28621 consensus_queue.cc:1243] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer 0ee4ab0747914e7abc642312517b5e25 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:32.178732 23677 catalog_manager.cc:5697] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.180979 23677 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.182977 23677 catalog_manager.cc:5697] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 14 to 15, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.188709 23677 catalog_manager.cc:5697] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.200614 23677 catalog_manager.cc:5697] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 8 to 13, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 13 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.206607 23781 pending_rounds.cc:85] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Aborting all ops after (but not including) 9
I20260709 12:15:32.207044 23781 pending_rounds.cc:107] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:15:32.207293 23781 raft_consensus.cc:2889] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:32.216770 23682 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 278) took 41673 ms (client timeout 15000 ms). Trace:
I20260709 12:15:32.212778 23677 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 7 to 10, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 10 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.265008 23677 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 12, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 12 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.267298 23677 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 4 to 6, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 5 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.283316 23927 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:15:32.285506 28621 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:32.223142 23682 rpcz_store.cc:269] 0709 12:14:50.543545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.543863 (+   318us) service_pool.cc:224] Handling call
0709 12:15:32.216556 (+41672693us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:31.364001 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:31.364670 (+   669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:31.364725 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:15:31.364743 (+    18us) tablet.cc:662] Decoding operations
0709 12:15:31.365164 (+   421us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:31.365257 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:31.365285 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:15:31.365624 (+   339us) tablet.cc:701] Row locks acquired
0709 12:15:31.365683 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:15:31.365865 (+   182us) write_op.cc:270] Start()
0709 12:15:31.366221 (+   356us) write_op.cc:276] Timestamp: P: 1783599331365798 usec, L: 0
0709 12:15:31.366260 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:15:31.367058 (+   798us) log.cc:844] Serialized 538 byte log entry
0709 12:15:31.820360 (+453302us) op_driver.cc:464] REPLICATION: finished
0709 12:15:31.985735 (+165375us) write_op.cc:301] APPLY: starting
0709 12:15:31.985861 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:32.011095 (+ 25234us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:32.011216 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:32.031795 (+ 20579us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:32.032075 (+   280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:15:32.032136 (+    61us) write_op.cc:312] APPLY: finished
0709 12:15:32.032435 (+   299us) log.cc:844] Serialized 25 byte log entry
0709 12:15:32.032639 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.032920 (+   281us) write_op.cc:454] Released schema lock
0709 12:15:32.032979 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:32.033481 (+   502us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.033510 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":627409,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":136446,"apply.run_cpu_time_us":5189,"apply.run_wall_time_us":61398,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":182326,"prepare.run_cpu_time_us":51304,"prepare.run_wall_time_us":119833,"raft.queue_time_us":170787,"raft.run_cpu_time_us":5461,"raft.run_wall_time_us":88541,"replication_time_us":443964,"thread_start_us":384819,"threads_started":4,"wal-append.queue_time_us":233785}]]}
I20260709 12:15:32.306942 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:32.307696 25170 raft_consensus.cc:606] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:32.308019 25170 raft_consensus.cc:614] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:32.311443 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:32.312089 23857 raft_consensus.cc:606] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:32.322659 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98e5bc86804f4f979f58779caa696814"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:32.323287 23927 raft_consensus.cc:606] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:32.323648 23927 raft_consensus.cc:614] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:32.327821 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:32.328414 23857 raft_consensus.cc:606] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:32.328770 23857 raft_consensus.cc:614] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:32.334030 25170 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:32.334664 25170 raft_consensus.cc:606] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:32.342752 25168 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:50586
I20260709 12:15:32.343381 25168 raft_consensus.cc:606] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:32.343720 25168 raft_consensus.cc:614] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:32.347384 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:32.349416 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:32.375648 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:32.386073 29408 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:32.386929 29408 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.390741 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:32.391570 29408 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:32.393370 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:32.396656 23749 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:32.399456 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:32.400746 29408 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:32.439249 23927 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:32.440749 29582 raft_consensus.cc:993] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:32.441267 29582 raft_consensus.cc:1081] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:32.441044 29390 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:32.444958 23854 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46448
I20260709 12:15:32.445662 23854 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.446008 23854 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:32.458539 23857 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:32.460203 23854 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.474725 23854 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:32.475853 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:32.478451 25170 raft_consensus.cc:3055] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:32.478765 25170 raft_consensus.cc:740] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:32.461117 29654 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:15:32.479574 25170 consensus_queue.cc:260] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.480773 25170 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:32.476913 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:32.483060 23927 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:32.501742 23927 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 19.
I20260709 12:15:32.521304 23824 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:32.522519 28621 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:32.522979 28621 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:32.524102 28621 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.572314 25170 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 19.
W20260709 12:15:32.641158 29688 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:32.664958 29604 raft_consensus.cc:993] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:32.665820 29604 raft_consensus.cc:1081] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:32.669690 23677 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 171) took 4890 ms. Trace:
I20260709 12:15:32.670222 23677 rpcz_store.cc:276] 0709 12:15:27.787019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.850134 (+ 63115us) service_pool.cc:224] Handling call
0709 12:15:32.669571 (+4819437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:32.397220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:32.397764 (+   544us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:32.397794 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:15:32.397811 (+    17us) tablet.cc:662] Decoding operations
0709 12:15:32.398724 (+   913us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:32.398800 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:32.398832 (+    32us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:15:32.399519 (+   687us) tablet.cc:701] Row locks acquired
0709 12:15:32.399533 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:15:32.399718 (+   185us) write_op.cc:270] Start()
0709 12:15:32.399824 (+   106us) write_op.cc:276] Timestamp: P: 1783599332399668 usec, L: 0
0709 12:15:32.399840 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:15:32.400548 (+   708us) log.cc:844] Serialized 3570 byte log entry
0709 12:15:32.451028 (+ 50480us) op_driver.cc:464] REPLICATION: finished
0709 12:15:32.490972 (+ 39944us) write_op.cc:301] APPLY: starting
0709 12:15:32.491075 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:32.498413 (+  7338us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:32.498490 (+    77us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:32.504048 (+  5558us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:32.504299 (+   251us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:15:32.504345 (+    46us) write_op.cc:312] APPLY: finished
0709 12:15:32.504617 (+   272us) log.cc:844] Serialized 89 byte log entry
0709 12:15:32.505638 (+  1021us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.505775 (+   137us) write_op.cc:454] Released schema lock
0709 12:15:32.505820 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:32.506269 (+   449us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.506297 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16538,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":1644,"apply.run_cpu_time_us":12306,"apply.run_wall_time_us":15262,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37638,"num_ops":9,"prepare.queue_time_us":71851,"prepare.run_cpu_time_us":3648,"prepare.run_wall_time_us":3648,"replication_time_us":51000,"thread_start_us":111790,"threads_started":1}]]}
I20260709 12:15:32.695369 23680 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:32.710737 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:32.711438 23781 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.711835 23781 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.763409 23680 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:32.782635 23781 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:32.785050 23781 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:32.785816 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:32.786466 23857 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:32.786779 23857 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:32.787547 23857 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:32.789474 23857 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.805366 23857 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:32.806921 23751 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:32.807955 29654 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:32.808498 29654 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:32.809490 29654 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:32.819708 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:32.821354 23927 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.828166 29469 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.828765 29469 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:32.831000 29469 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:32.832006 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:32.832902 23783 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:15:32.834362 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet de3c70defd864769b26b52b7f36548cd with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:32.834733 23927 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:32.835549 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:32.836455 23858 raft_consensus.cc:2410] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:15:32.837509 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:32.839066 29469 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:32.840775 23677 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.842541 23677 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.844403 23677 catalog_manager.cc:5697] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:32.846040 23677 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 15, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.863288 29693 tablet_replica.cc:333] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:15:32.876995 23677 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 13, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.879405 23677 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.880957 23677 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 16, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.882632 23677 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 16 to 17, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 17 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.884287 23677 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 13, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.885916 23677 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 14 to 17, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 17 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:32.889612 23680 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 271) took 30098 ms (client timeout 15000 ms). Trace:
W20260709 12:15:32.890038 23680 rpcz_store.cc:269] 0709 12:15:02.791104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.792172 (+  1068us) service_pool.cc:224] Handling call
0709 12:15:02.856123 (+ 63951us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:32.889526 (+30033403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:32.797920 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:32.798453 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:32.798491 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:15:32.798509 (+    18us) tablet.cc:662] Decoding operations
0709 12:15:32.798890 (+   381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:32.798945 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:32.798973 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:15:32.799301 (+   328us) tablet.cc:701] Row locks acquired
0709 12:15:32.799314 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:15:32.799457 (+   143us) write_op.cc:270] Start()
0709 12:15:32.799563 (+   106us) write_op.cc:276] Timestamp: P: 1783599332799411 usec, L: 0
0709 12:15:32.799583 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:15:32.800425 (+   842us) log.cc:844] Serialized 923 byte log entry
0709 12:15:32.802108 (+  1683us) op_driver.cc:464] REPLICATION: finished
0709 12:15:32.802573 (+   465us) write_op.cc:301] APPLY: starting
0709 12:15:32.802659 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:32.804061 (+  1402us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:32.804164 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:32.805410 (+  1246us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:32.805607 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:15:32.805662 (+    55us) write_op.cc:312] APPLY: finished
0709 12:15:32.806198 (+   536us) log.cc:844] Serialized 33 byte log entry
0709 12:15:32.806382 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.806472 (+    90us) write_op.cc:454] Released schema lock
0709 12:15:32.806524 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:32.806963 (+   439us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:32.806991 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"spinlock_wait_cycles":40579328,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":4039,"apply.run_wall_time_us":4332,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":635,"prepare.run_cpu_time_us":2756,"prepare.run_wall_time_us":2807,"replication_time_us":2352,"spinlock_wait_cycles":380160}]]}
I20260709 12:15:32.897414 29693 raft_consensus.cc:2243] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:32.898595 29693 raft_consensus.cc:2272] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:32.900166 23677 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 15 to 16, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:32.905490 29693 ts_tablet_manager.cc:1916] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:32.974866 23927 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:32.976534 29604 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:32.979316 29693 ts_tablet_manager.cc:1929] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:15:32.997000 29693 log.cc:1199] T de3c70defd864769b26b52b7f36548cd P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/de3c70defd864769b26b52b7f36548cd
I20260709 12:15:33.015127 25170 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:33.021566 28621 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:33.054683 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 7 to 9, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 9 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.057077 23683 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 13 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.058662 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet de3c70defd864769b26b52b7f36548cd (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:33.059526 23683 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 12 to 14, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.072444 29654 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.091498 23683 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 14, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.089781 29654 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.113252 29654 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:33.114037 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:33.114821 23858 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:33.115967 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:33.117779 23683 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.120250 23683 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 12, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.122251 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 18, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 18 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.124274 23683 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 7 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.126051 23683 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.127614 23683 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.130810 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:33.131543 25174 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:33.134850 29654 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:33.135365 29654 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.135752 29654 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.149837 29654 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.128916 23683 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 12, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 12 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.160210 23683 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 14, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.162045 23683 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 12 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.165160 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 9 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.172053 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:33.178017 23858 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.174208 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:33.183993 25174 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.201902 25174 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:33.200238 23858 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:33.207446 29654 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:33.208755 23749 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:15:33.210275 29654 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:33.204840 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 11, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.215726 29654 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:33.216790 29654 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.295804 23677 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 15 to 19, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 19 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:33.349462 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 276) took 8552 ms. Trace:
I20260709 12:15:33.353683 23676 catalog_manager.cc:5697] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 16 to 17, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.364701 23933 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:33.366549 29654 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:33.364399 23676 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 13 to 19, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 19 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.376035 23676 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 13 to 18, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 18 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.360231 23683 rpcz_store.cc:276] 0709 12:15:24.797316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.802289 (+  4973us) service_pool.cc:224] Handling call
0709 12:15:24.850677 (+ 48388us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:33.349370 (+8498693us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:33.230190 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:33.231070 (+   880us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:33.231110 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:15:33.231123 (+    13us) tablet.cc:662] Decoding operations
0709 12:15:33.232492 (+  1369us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:33.232547 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:33.232579 (+    32us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:15:33.233697 (+  1118us) tablet.cc:701] Row locks acquired
0709 12:15:33.233716 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:15:33.233887 (+   171us) write_op.cc:270] Start()
0709 12:15:33.234001 (+   114us) write_op.cc:276] Timestamp: P: 1783599333233835 usec, L: 0
0709 12:15:33.234025 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:15:33.234967 (+   942us) log.cc:844] Serialized 5929 byte log entry
0709 12:15:33.238163 (+  3196us) op_driver.cc:464] REPLICATION: finished
0709 12:15:33.238747 (+   584us) write_op.cc:301] APPLY: starting
0709 12:15:33.238833 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:33.247511 (+  8678us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:33.247623 (+   112us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:33.280401 (+ 32778us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:33.280662 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:15:33.280710 (+    48us) write_op.cc:312] APPLY: finished
0709 12:15:33.281032 (+   322us) log.cc:844] Serialized 139 byte log entry
0709 12:15:33.289819 (+  8787us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.289978 (+   159us) write_op.cc:454] Released schema lock
0709 12:15:33.290033 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:33.290470 (+   437us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.309937 (+ 19467us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12587,"reported_tablets":50,"spinlock_wait_cycles":8513792,"child_traces":[["op",{"apply.queue_time_us":383,"apply.run_cpu_time_us":19667,"apply.run_wall_time_us":51669,"cfile_cache_hit":60,"cfile_cache_hit_bytes":62730,"num_ops":15,"prepare.queue_time_us":150,"prepare.run_cpu_time_us":4703,"prepare.run_wall_time_us":7409,"replication_time_us":3984}]]}
I20260709 12:15:33.380983 23676 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 10 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.384429 23676 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 11 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.391166 23676 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.401414 23676 catalog_manager.cc:5697] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 9 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.406157 23676 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 10 to 14, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.432685 23858 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:33.435307 23676 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 11 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.436184 29536 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:33.449427 23676 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 11 to 12, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 12 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.518391 23676 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 7 to 13, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.520354 23676 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.522082 23676 catalog_manager.cc:5697] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 13 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.560171 23676 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 9 to 10, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 10 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.567878 23676 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 14 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.669332 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:33.724257 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 44261.9us
I20260709 12:15:33.778455 23676 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 280) took 4731 ms. Trace:
I20260709 12:15:33.781927 23681 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:33.782996 23676 rpcz_store.cc:276] 0709 12:15:29.047199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.047456 (+   257us) service_pool.cc:224] Handling call
0709 12:15:29.645001 (+597545us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400002490. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2aa54ddb1 00007fb2aa68e640 00007fb2b3565846 00007fb2b366f90e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:33.778374 (+4133373us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:33.606041 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:33.606587 (+   546us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:33.606623 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:15:33.606634 (+    11us) tablet.cc:662] Decoding operations
0709 12:15:33.607955 (+  1321us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:33.608012 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:33.608046 (+    34us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:15:33.609151 (+  1105us) tablet.cc:701] Row locks acquired
0709 12:15:33.609171 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:15:33.609326 (+   155us) write_op.cc:270] Start()
0709 12:15:33.609446 (+   120us) write_op.cc:276] Timestamp: P: 1783599333609278 usec, L: 0
0709 12:15:33.609467 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:15:33.610233 (+   766us) log.cc:844] Serialized 5925 byte log entry
0709 12:15:33.612786 (+  2553us) op_driver.cc:464] REPLICATION: finished
0709 12:15:33.613264 (+   478us) write_op.cc:301] APPLY: starting
0709 12:15:33.613342 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:33.630810 (+ 17468us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:33.630912 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:33.672502 (+ 41590us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:33.672779 (+   277us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:15:33.672834 (+    55us) write_op.cc:312] APPLY: finished
0709 12:15:33.673174 (+   340us) log.cc:844] Serialized 139 byte log entry
0709 12:15:33.673356 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.673520 (+   164us) write_op.cc:454] Released schema lock
0709 12:15:33.673573 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:33.707896 (+ 34323us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.707927 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":265596,"reported_tablets":49,"spinlock_wait_cycles":67086720,"child_traces":[["op",{"apply.queue_time_us":263,"apply.run_cpu_time_us":20208,"apply.run_wall_time_us":94550,"cfile_cache_hit":60,"cfile_cache_hit_bytes":62730,"num_ops":15,"prepare.queue_time_us":6372,"prepare.run_cpu_time_us":4522,"prepare.run_wall_time_us":4522,"replication_time_us":3194,"spinlock_wait_cycles":77184}]]}
I20260709 12:15:33.787268 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:33.819815 23927 raft_consensus.cc:606] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:33.830556 23858 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:33.834580 29408 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:33.843175 23854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:33.843930 23854 raft_consensus.cc:606] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:33.858614 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:33.859254 23927 raft_consensus.cc:606] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:33.864287 23927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:33.864893 23927 raft_consensus.cc:606] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:33.867713 23859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:33.868366 23859 raft_consensus.cc:606] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:33.865274 25174 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:33.870568 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:33.875485 29533 raft_consensus.cc:993] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:33.885069 29533 raft_consensus.cc:1081] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:33.893162 23859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:33.893816 23859 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:33.894200 23859 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:33.887866 29408 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:33.901139 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:33.902088 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:33.917119 23859 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.923182 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:33.923854 23927 raft_consensus.cc:3055] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:15:33.924185 23927 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:33.925112 23927 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.926795 23927 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:33.933054 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:33.933746 25170 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:33.942693 25170 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:15:33.943850 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:33.944727 29501 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:15:33.945259 29501 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:33.946218 29501 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:33.951013 23859 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 6 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:34.046551 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 87105us
I20260709 12:15:34.058880 23927 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
W20260709 12:15:34.133600 29710 log.cc:927] Time spent T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
I20260709 12:15:34.149430 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 272) took 11107 ms. Trace:
I20260709 12:15:34.149899 23681 rpcz_store.cc:276] 0709 12:15:23.041755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.042022 (+   267us) service_pool.cc:224] Handling call
0709 12:15:34.149351 (+11107329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:33.795855 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:33.796393 (+   538us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:33.796427 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:15:33.796445 (+    18us) tablet.cc:662] Decoding operations
0709 12:15:33.796816 (+   371us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:33.796870 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:33.796899 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:15:33.797077 (+   178us) tablet.cc:701] Row locks acquired
0709 12:15:33.797100 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:15:33.797232 (+   132us) write_op.cc:270] Start()
0709 12:15:33.797340 (+   108us) write_op.cc:276] Timestamp: P: 1783599333797188 usec, L: 0
0709 12:15:33.797362 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:15:33.797978 (+   616us) log.cc:844] Serialized 538 byte log entry
0709 12:15:33.799729 (+  1751us) op_driver.cc:464] REPLICATION: finished
0709 12:15:33.800175 (+   446us) write_op.cc:301] APPLY: starting
0709 12:15:33.800248 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:33.801090 (+   842us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:33.801196 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:33.801936 (+   740us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:33.802120 (+   184us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:15:33.802174 (+    54us) write_op.cc:312] APPLY: finished
0709 12:15:33.802360 (+   186us) log.cc:844] Serialized 25 byte log entry
0709 12:15:33.802523 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.802602 (+    79us) write_op.cc:454] Released schema lock
0709 12:15:33.802640 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:33.803034 (+   394us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.803060 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":755794,"reported_tablets":50,"spinlock_wait_cycles":14976,"child_traces":[["op",{"apply.queue_time_us":234,"apply.run_cpu_time_us":2808,"apply.run_wall_time_us":2808,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":4881,"prepare.run_cpu_time_us":2405,"prepare.run_wall_time_us":2405,"replication_time_us":2201}]]}
I20260709 12:15:34.204140 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet de3c70defd864769b26b52b7f36548cd with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:34.308166 23679 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:34.388051 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 180241us
W20260709 12:15:34.406141 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 114831us
W20260709 12:15:34.416319 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 116449us
W20260709 12:15:34.428130 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 211118us
W20260709 12:15:34.496537 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 66605.7us
I20260709 12:15:34.525512 29660 raft_consensus.cc:993] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:34.526209 29660 raft_consensus.cc:1081] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:34.546844 29501 raft_consensus.cc:993] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:34.547782 29501 raft_consensus.cc:1081] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:15:34.636413 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 188205us
I20260709 12:15:34.663820 23781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:34.664523 23781 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.664913 23781 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.668702 23859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:34.669425 23859 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.669818 23859 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:34.676803 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 242999us
I20260709 12:15:34.693100 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet de3c70defd864769b26b52b7f36548cd (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:34.697299 23859 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:34.699275 28621 raft_consensus.cc:993] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:34.700852 28621 raft_consensus.cc:1081] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:34.704535 23781 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.704329 23859 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:34.719794 23859 raft_consensus.cc:1240] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:34.726624 29719 consensus_queue.cc:1059] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:34.728363 29716 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:34.728788 29716 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:34.729635 29716 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:34.731114 29716 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:34.759176 23781 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:34.760536 23933 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:15:34.764802 23784 raft_consensus.cc:1240] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:34.764802 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:34.766592 23777 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.766963 23777 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.775803 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:34.782052 23777 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.855832 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:34.856524 23927 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.870390 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:34.871033 23851 raft_consensus.cc:3055] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:34.871335 23851 raft_consensus.cc:740] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:34.872234 23777 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:34.872933 29711 raft_consensus.cc:993] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:34.873494 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:34.888193 25174 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:34.876111 25168 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:15:34.875982 29501 consensus_queue.cc:1059] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:15:34.895481 29711 raft_consensus.cc:1081] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:34.895967 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:34.896608 23930 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.904264 23783 raft_consensus.cc:1240] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:34.905479 23784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:15:34.906087 23784 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:34.906414 23784 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:34.906713 25174 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:34.920475 23784 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.922276 23927 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:34.923607 23749 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:34.925338 29740 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:34.925869 29740 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:34.926805 29740 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.929765 29743 consensus_queue.cc:1059] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:34.930864 29743 raft_consensus.cc:3055] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:34.931249 29743 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:34.932451 23851 consensus_queue.cc:260] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.933648 23851 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.939311 29743 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.941039 29743 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.941776 23824 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:34.941624 29726 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:34.945294 23926 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:34.948823 29755 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:34.949412 29755 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:34.950321 29755 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:34.966732 29742 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:34.976267 23784 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:34.977406 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:34.984231 23926 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:34.984639 23926 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:34.985559 23926 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:34.986822 23926 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:34.978176 23851 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:34.978518 23930 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:34.998888 23749 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:35.000429 29721 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:35.000996 29721 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:35.002048 29721 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:35.009519 23926 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 19.
I20260709 12:15:35.016116 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:35.019987 23749 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:35.021757 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:35.022392 23859 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:35.023413 29712 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:35.022922 23676 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 14, leader changed from 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:35.024049 29712 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:35.024950 29712 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:35.052459 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 10 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.057569 23683 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.076841 23851 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:35.082060 23683 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.088069 23859 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 19.
I20260709 12:15:35.124485 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:35.142760 23683 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 10 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.144308 23683 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.147298 23683 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 13 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:35.149282 23683 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 14 to 16, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:15:35.293963 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 136534us
I20260709 12:15:35.306267 23676 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 175) took 1239 ms. Trace:
I20260709 12:15:35.306768 23676 rpcz_store.cc:276] 0709 12:15:34.066806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.078150 (+ 11344us) service_pool.cc:224] Handling call
0709 12:15:35.306161 (+1228011us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:35.026659 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:35.032576 (+  5917us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:35.032616 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:15:35.032632 (+    16us) tablet.cc:662] Decoding operations
0709 12:15:35.032987 (+   355us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:35.033041 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:35.033072 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:15:35.033272 (+   200us) tablet.cc:701] Row locks acquired
0709 12:15:35.033288 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:15:35.033438 (+   150us) write_op.cc:270] Start()
0709 12:15:35.033551 (+   113us) write_op.cc:276] Timestamp: P: 1783599335033389 usec, L: 0
0709 12:15:35.033572 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:15:35.034222 (+   650us) log.cc:844] Serialized 533 byte log entry
0709 12:15:35.088611 (+ 54389us) op_driver.cc:464] REPLICATION: finished
0709 12:15:35.093207 (+  4596us) write_op.cc:301] APPLY: starting
0709 12:15:35.093331 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:35.094281 (+   950us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:35.094394 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:35.095409 (+  1015us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:35.095556 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:15:35.109386 (+ 13830us) write_op.cc:312] APPLY: finished
0709 12:15:35.109606 (+   220us) log.cc:844] Serialized 25 byte log entry
0709 12:15:35.109791 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.109876 (+    85us) write_op.cc:454] Released schema lock
0709 12:15:35.109917 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:35.110278 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.110305 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36197,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":2108,"apply.run_cpu_time_us":3360,"apply.run_wall_time_us":17075,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":674,"prepare.run_cpu_time_us":2646,"prepare.run_wall_time_us":7873,"replication_time_us":52717}]]}
W20260709 12:15:35.431433 23679 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 279) took 23687 ms (client timeout 15000 ms). Trace:
W20260709 12:15:35.431937 23679 rpcz_store.cc:269] 0709 12:15:11.743787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.751717 (+  7930us) service_pool.cc:224] Handling call
0709 12:15:35.431332 (+23679615us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:34.695806 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:34.696387 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:34.696424 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:15:34.696443 (+    19us) tablet.cc:662] Decoding operations
0709 12:15:34.696777 (+   334us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:34.696832 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:34.696864 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:15:34.697048 (+   184us) tablet.cc:701] Row locks acquired
0709 12:15:34.697064 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:15:34.697217 (+   153us) write_op.cc:270] Start()
0709 12:15:34.697333 (+   116us) write_op.cc:276] Timestamp: P: 1783599334697171 usec, L: 0
0709 12:15:34.697356 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:15:34.698325 (+   969us) log.cc:844] Serialized 538 byte log entry
0709 12:15:34.774950 (+ 76625us) op_driver.cc:464] REPLICATION: finished
0709 12:15:35.000381 (+225431us) write_op.cc:301] APPLY: starting
0709 12:15:35.000479 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:35.001775 (+  1296us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:35.001893 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:35.002671 (+   778us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:35.002930 (+   259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:15:35.002981 (+    51us) write_op.cc:312] APPLY: finished
0709 12:15:35.003200 (+   219us) log.cc:844] Serialized 25 byte log entry
0709 12:15:35.003403 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.003491 (+    88us) write_op.cc:454] Released schema lock
0709 12:15:35.003531 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:35.003981 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.004020 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90998,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":225154,"apply.run_cpu_time_us":3375,"apply.run_wall_time_us":3577,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":240461,"prepare.run_cpu_time_us":2714,"prepare.run_wall_time_us":49272,"replication_time_us":77416,"thread_start_us":442647,"threads_started":2}]]}
W20260709 12:15:35.486075 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 75691us
W20260709 12:15:35.473106 29649 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:15:35.503569 23930 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:35.507023 23926 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:35.524080 23851 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:35.532050 29712 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:35.539651 23926 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:35.541177 23926 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:35.544667 29501 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:35.562026 23851 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:15:35.567617 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 223640us
I20260709 12:15:35.587288 23933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:35.593501 25170 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:35.572600 29654 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:15:35.588531 23933 raft_consensus.cc:606] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:35.584571 29771 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:15:35.612421 29721 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:35.648281 29720 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:35.675092 29726 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:15:35.731738 23679 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 5 to 6, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 6 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:35.750492 23858 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:35.750988 23859 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:35.774055 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62141.4us
W20260709 12:15:35.777724 29783 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:15:35.793233 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 277) took 2222 ms. Trace:
I20260709 12:15:35.766494 29654 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:15:35.819315 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:35.820092 23857 raft_consensus.cc:606] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:35.793789 23683 rpcz_store.cc:276] 0709 12:15:33.551581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:33.551834 (+   253us) service_pool.cc:224] Handling call
0709 12:15:35.773853 (+2222019us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:35.332284 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:35.332867 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:35.332904 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:15:35.332918 (+    14us) tablet.cc:662] Decoding operations
0709 12:15:35.333794 (+   876us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:35.333858 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:35.333886 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:15:35.334551 (+   665us) tablet.cc:701] Row locks acquired
0709 12:15:35.334571 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:15:35.334721 (+   150us) write_op.cc:270] Start()
0709 12:15:35.334844 (+   123us) write_op.cc:276] Timestamp: P: 1783599335334666 usec, L: 0
0709 12:15:35.334866 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:15:35.395219 (+ 60353us) log.cc:844] Serialized 2849 byte log entry
0709 12:15:35.516645 (+121426us) op_driver.cc:464] REPLICATION: finished
0709 12:15:35.517113 (+   468us) write_op.cc:301] APPLY: starting
0709 12:15:35.517197 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:35.531652 (+ 14455us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:35.531776 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:35.645078 (+113302us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:35.645299 (+   221us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:15:35.645347 (+    48us) write_op.cc:312] APPLY: finished
0709 12:15:35.645603 (+   256us) log.cc:844] Serialized 73 byte log entry
0709 12:15:35.645782 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.645896 (+   114us) write_op.cc:454] Released schema lock
0709 12:15:35.645942 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:35.646292 (+   350us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.646320 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":101378,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":254,"apply.run_cpu_time_us":10801,"apply.run_wall_time_us":129154,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29274,"num_ops":7,"prepare.queue_time_us":8917,"prepare.run_cpu_time_us":4177,"prepare.run_wall_time_us":72357,"replication_time_us":181474}]]}
I20260709 12:15:35.917591 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:35.918226 23857 raft_consensus.cc:606] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
W20260709 12:15:35.939559 29784 log.cc:927] Time spent T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:15:36.052891 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 114466us
W20260709 12:15:36.081120 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 143964us
W20260709 12:15:36.103961 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 77524.7us
I20260709 12:15:36.134708 25168 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:36.148468 29777 raft_consensus.cc:993] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:36.149003 29777 raft_consensus.cc:1081] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:36.150764 23858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:36.151301 23858 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:36.151710 23858 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.165547 23857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:36.166275 23857 raft_consensus.cc:606] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:36.194410 23859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:36.195190 23859 raft_consensus.cc:606] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:36.195538 23859 raft_consensus.cc:614] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:36.205286 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:36.194247 23858 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.225806 23858 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:36.239576 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:36.240533 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:36.239925 23683 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 12 to 13, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:36.244237 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 47073.4us
I20260709 12:15:36.251863 29743 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:36.267017 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:36.267704 23926 raft_consensus.cc:3055] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:36.267987 23926 raft_consensus.cc:740] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:36.268718 23926 consensus_queue.cc:260] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.269994 23926 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.273068 23681 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58556 (request call id 281) took 1839 ms. Trace:
I20260709 12:15:36.273496 23681 rpcz_store.cc:276] 0709 12:15:34.433394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.433509 (+   115us) service_pool.cc:224] Handling call
0709 12:15:36.272998 (+1839489us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13162,"reported_tablets":21}
I20260709 12:15:36.283429 23784 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:36.296545 23926 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:36.298274 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:36.298959 23777 raft_consensus.cc:3060] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.320755 23824 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:36.329373 29742 raft_consensus.cc:2804] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:36.345278 23777 raft_consensus.cc:2468] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
I20260709 12:15:36.355836 29742 raft_consensus.cc:697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:36.357143 29742 consensus_queue.cc:237] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.385872 29803 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 7 to 14, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:36.387776 29802 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:36.387235 25168 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 7 to 14, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:36.417574 29796 raft_consensus.cc:993] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.417690 29501 raft_consensus.cc:993] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.418174 29796 raft_consensus.cc:1081] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.418710 29501 raft_consensus.cc:1081] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.475181 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6d3d4f634284532bbf6ad7d4ac19214 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:36.481974 23677 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58540 (request call id 273) took 1651 ms. Trace:
I20260709 12:15:36.485143 23677 rpcz_store.cc:276] 0709 12:15:34.830191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.830391 (+   200us) service_pool.cc:224] Handling call
0709 12:15:35.002043 (+171652us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:15:36.481888 (+1479845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22816,"reported_tablets":36,"spinlock_wait_cycles":343838464}
I20260709 12:15:36.504429 23677 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 14 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:36.510910 23677 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 14 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:36.515717 23677 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 18 to 19, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 19 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:36.523154 29802 raft_consensus.cc:993] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.523764 29803 raft_consensus.cc:1081] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:36.594866 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:36.596374 23782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:36.584275 23683 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 7 to 14, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:36.619789 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:36.620553 23780 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:36.622856 23780 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.629755 23782 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:36.630280 23782 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:36.634428 23779 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:36.634824 23779 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:36.656690 23779 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.666278 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:36.666922 23856 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:36.667212 23856 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:36.668036 23856 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.671643 23856 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:36.690819 23784 raft_consensus.cc:2955] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 7 to 14, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:36.689880 23780 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.705049 23777 raft_consensus.cc:1240] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:36.709544 23780 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:36.710804 23853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:36.712030 23853 raft_consensus.cc:3055] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:36.712340 23853 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:36.690675 23782 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:36.712052 23779 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:36.714093 23853 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.715082 23782 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:36.715936 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:36.715299 23853 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.716539 23858 raft_consensus.cc:3055] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:36.716982 23858 raft_consensus.cc:740] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:36.717762 23858 consensus_queue.cc:260] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:36.718961 23858 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:36.720045 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet d6d3d4f634284532bbf6ad7d4ac19214 with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:50860
I20260709 12:15:36.722959 23681 catalog_manager.cc:5697] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:36.727629 23858 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:36.729219 23751 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:36.730473 29536 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:36.731029 29536 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:36.731963 29536 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:36.752032 23856 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:36.758909 23751 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:36.761209 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:36.761870 23932 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.762712 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:36.763001 29813 tablet_replica.cc:333] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:15:36.761229 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:36.763767 25170 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:36.759799 29771 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:36.768218 29771 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:36.752342 23853 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
W20260709 12:15:36.754690 23825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.769186 29771 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.772985 23751 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:36.774057 29712 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:36.774595 29712 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:36.775477 29712 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:36.778858 23926 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:36.796949 23784 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:36.798444 23926 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 15.
I20260709 12:15:36.795275 23932 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:36.798533 25170 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:36.808219 29780 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:36.812898 29813 raft_consensus.cc:2243] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:36.814128 29813 raft_consensus.cc:2272] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:36.820573 29813 ts_tablet_manager.cc:1916] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:36.857005 23681 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 16 to 17, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 17 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:36.860109 29813 ts_tablet_manager.cc:1929] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.13
I20260709 12:15:36.862555 23926 raft_consensus.cc:1275] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:36.872677 29813 log.cc:1199] T d6d3d4f634284532bbf6ad7d4ac19214 P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/d6d3d4f634284532bbf6ad7d4ac19214
I20260709 12:15:36.873392 29803 consensus_queue.cc:1048] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:36.924587 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet d6d3d4f634284532bbf6ad7d4ac19214 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:36.935413 23681 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 14 to 15, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:36.938532 23681 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 15 to 16, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:37.018217 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 64273.9us
I20260709 12:15:37.184357 23858 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:15:37.186080 29536 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:37.189742 25168 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:15:37.191627 29801 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:15:37.253573 23928 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:37.256671 29730 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:37.275470 23926 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:37.277523 29801 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:37.320662 23858 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:37.323133 29730 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:37.323557 23858 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:37.326563 29730 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:37.584649 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:37.826386 23779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:37.827020 23779 raft_consensus.cc:606] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:37.835240 23858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:37.835924 23858 raft_consensus.cc:606] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:37.842157 23779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:37.842797 23779 raft_consensus.cc:606] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:37.845507 23779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:37.846102 23779 raft_consensus.cc:606] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:37.858279 23779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:37.881580 23779 raft_consensus.cc:606] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:37.885186 23779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:37.885910 23779 raft_consensus.cc:606] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:37.887938 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:37.889860 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:37.890659 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:37.890844 29801 raft_consensus.cc:993] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:37.893851 29712 raft_consensus.cc:1081] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:37.895715 23928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:37.896278 23928 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:37.896631 23928 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:37.913968 23926 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:37.921810 29712 raft_consensus.cc:993] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:37.922428 29712 raft_consensus.cc:1081] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:37.925424 23932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:37.925575 23928 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:37.926009 23932 raft_consensus.cc:493] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:37.926365 23932 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:37.928617 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:37.929289 23779 raft_consensus.cc:3055] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:37.929551 23779 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:37.930660 23928 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:37.934005 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:37.934618 23858 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:37.930299 23779 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:37.942414 23898 consensus_peers.cc:597] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6f2d1f3ed104c1e99f82ebe04843248. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:37.944428 23857 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:37.945916 23784 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:37.938486 23779 raft_consensus.cc:3060] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:37.947683 29716 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:37.949679 29716 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:38.037981 29756 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:38.045921 23856 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:38.068868 29822 raft_consensus.cc:993] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:38.069415 29822 raft_consensus.cc:1081] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:38.072903 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:38.073491 23777 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:38.073853 23777 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:38.075930 29712 raft_consensus.cc:993] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.076404 29730 raft_consensus.cc:993] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.076949 29730 raft_consensus.cc:1081] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.077437 29712 raft_consensus.cc:1081] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.097668 23932 raft_consensus.cc:515] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:38.149885 23932 leader_election.cc:290] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:38.160130 23779 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
W20260709 12:15:38.164270 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 112381us
I20260709 12:15:38.178947 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:38.114058 23858 raft_consensus.cc:2468] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:15:38.118829 23777 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.202795 23777 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:38.205430 23900 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.206769 29756 raft_consensus.cc:2804] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:38.207358 29756 raft_consensus.cc:697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:38.209972 29756 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.213589 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:38.214308 23858 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:38.214674 23858 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:38.216094 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:38.216732 25168 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:38.216845 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:38.219200 23926 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:38.219548 23926 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.228088 23926 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:38.230157 23926 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:38.239940 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:38.240602 23928 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:38.246958 25168 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:38.253074 25170 raft_consensus.cc:1240] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:38.253631 23928 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:38.256402 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:38.257030 25170 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.259804 29712 consensus_queue.cc:1059] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:38.261178 29730 raft_consensus.cc:3055] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:38.261564 29730 raft_consensus.cc:740] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:38.262372 29730 consensus_queue.cc:260] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:38.266896 25170 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:38.239156 23932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:38.268303 23932 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:38.268697 23932 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.273293 23898 leader_election.cc:304] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.275297 29839 raft_consensus.cc:2804] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:38.262370 23858 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.275903 29839 raft_consensus.cc:697] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:38.276840 29839 consensus_queue.cc:237] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:38.281028 29730 raft_consensus.cc:3060] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:38.280926 23749 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.285861 29771 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:38.287106 29771 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:38.288059 29771 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.289481 23898 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.291247 29840 raft_consensus.cc:2804] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:38.293340 29840 raft_consensus.cc:697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:38.300202 23932 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.301403 23858 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:38.294296 29840 consensus_queue.cc:237] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:38.306010 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 233017us
I20260709 12:15:38.308197 23932 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:38.310822 23857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:38.313309 23857 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.323705 23858 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:15:38.336140 23857 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:38.337841 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.345285 29712 raft_consensus.cc:993] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.345968 29712 raft_consensus.cc:1081] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:38.339733 29716 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:38.359917 23932 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:38.360666 23932 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:38.361028 23932 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:38.361155 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bec362a6d01840d39be46f0c3d28c218" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:38.376439 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:38.377105 23777 raft_consensus.cc:3055] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:38.377391 23777 raft_consensus.cc:740] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:38.378113 23777 consensus_queue.cc:260] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:38.379360 23777 raft_consensus.cc:3060] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.381462 23784 raft_consensus.cc:2468] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:15:38.383023 23681 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 12 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:38.356765 29716 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:38.398658 23932 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.408630 23932 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 20 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:38.409718 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:38.410369 23858 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:38.413161 23777 raft_consensus.cc:2468] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:38.418969 23858 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 20.
W20260709 12:15:38.410643 29842 log.cc:927] Time spent T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:15:38.461195 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58183us
W20260709 12:15:38.518502 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 43682.6us
W20260709 12:15:38.543610 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 83740.9us
I20260709 12:15:38.461141 29716 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:38.550253 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 125969us
W20260709 12:15:38.601855 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 49734.4us
W20260709 12:15:38.609521 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46502.4us
I20260709 12:15:38.614285 23900 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:38.619015 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:38.615336 29840 raft_consensus.cc:2804] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:38.619714 23777 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:38.620523 23777 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:38.625063 23780 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Committing config change with OpId 11.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:38.634493 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:38.635253 23779 raft_consensus.cc:3055] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:38.635574 23779 raft_consensus.cc:740] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:38.636353 23779 consensus_queue.cc:260] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.637612 23779 raft_consensus.cc:3060] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:38.638202 29840 raft_consensus.cc:697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 20 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:38.639200 29840 consensus_queue.cc:237] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.652179 23777 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:38.686791 23857 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:38.692097 29716 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:38.695900 23777 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.699240 23928 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:38.700505 23779 raft_consensus.cc:2468] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 20.
I20260709 12:15:38.712823 25168 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:38.715044 29838 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:38.721769 23777 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:38.733839 23853 raft_consensus.cc:1240] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:38.736671 23859 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:38.741331 29801 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:38.747159 23777 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:38.748844 23776 raft_consensus.cc:1275] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:38.748641 23683 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } }
I20260709 12:15:38.749935 29838 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:38.752517 25170 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:15:38.757741 29875 log.cc:927] Time spent T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:15:38.788194 23683 catalog_manager.cc:5697] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 12 to 13, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:38.763813 29838 consensus_queue.cc:1048] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:15:38.801071 29876 ts_tablet_manager.cc:933] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:38.795231 23683 catalog_manager.cc:5697] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 17 to 18, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 18 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:38.805787 23683 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:38.764109 29840 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:15:38.807540 23683 catalog_manager.cc:5697] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 19 to 20, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 20 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:38.871929 23928 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:38.879524 29876 tablet_copy_client.cc:323] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:15:38.881785 29801 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:38.884048 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet c6f2d1f3ed104c1e99f82ebe04843248 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:15:38.884697 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet c6f2d1f3ed104c1e99f82ebe04843248 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-c6f2d1f3ed104c1e99f82ebe04843248
I20260709 12:15:38.892827 23776 raft_consensus.cc:1275] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:38.895944 29838 consensus_queue.cc:1048] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:38.940078 23944 tablet_copy_source_session.cc:215] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:38.945386 29876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f2d1f3ed104c1e99f82ebe04843248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:38.947739 23777 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:38.992601 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:39.002605 23780 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:39.010051 29876 tablet_copy_client.cc:806] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:39.016095 29838 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:39.016898 29876 tablet_copy_client.cc:670] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
W20260709 12:15:39.022895 23749 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b93ab9c24064567b717af205bc2ed2a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:39.029390 29876 tablet_copy_client.cc:538] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:39.040378 23859 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:39.043947 29730 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:39.045856 23926 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:39.060029 29843 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:15:39.088876 29876 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:39.094666 29801 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.097831 23853 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.120694 23859 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:39.122264 29838 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:39.137128 23859 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:39.140273 23932 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:39.141042 23932 raft_consensus.cc:606] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:39.155577 23777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:39.156351 23777 raft_consensus.cc:606] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:39.144157 29838 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:15:39.171013 23777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:39.171701 23777 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:39.176613 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:15:39.178735 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:39.179489 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:39.224342 23777 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:39.238175 23779 raft_consensus.cc:1275] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:39.245993 29738 consensus_queue.cc:1048] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:39.247936 29838 raft_consensus.cc:993] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:39.259255 23681 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.248399 29838 raft_consensus.cc:1081] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:15:39.276255 29890 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.063s	user 0.003s	sys 0.012s
W20260709 12:15:39.281836 29895 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.069s	user 0.003s	sys 0.005s
I20260709 12:15:39.330375 23857 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:39.335035 29730 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:39.346014 23927 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:15:39.342537 29894 log.cc:927] Time spent T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:15:39.379096 23776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:15:39.379719 23776 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:39.380007 23776 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:39.386201 29899 raft_consensus.cc:993] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:39.387982 29899 raft_consensus.cc:1081] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:39.389698 23776 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.391814 23776 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:39.403982 23857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:39.404597 23857 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:39.405022 23857 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:39.413304 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:39.414080 23927 raft_consensus.cc:3055] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:39.414393 23927 raft_consensus.cc:740] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:39.416949 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:39.417608 23856 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:39.415213 23927 consensus_queue.cc:260] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.419080 23927 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:39.427624 23927 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:39.423540 23857 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.434715 23857 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:39.437245 29712 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:39.437947 29712 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:39.441282 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:39.441911 23927 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:39.447659 23749 consensus_peers.cc:597] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5038ac294ceb4758b5b5de70f2469d5d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:39.448597 23856 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:39.450524 29843 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:39.451730 23751 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:39.451730 23856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:39.453678 29843 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:39.453902 23856 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:39.454867 23856 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:39.457603 29843 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:39.458873 29843 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.470697 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7caafee6779404494f8df62d888c5ef" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:39.471499 23776 raft_consensus.cc:3055] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:39.470700 23856 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:39.471982 23776 raft_consensus.cc:740] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:39.473316 23776 consensus_queue.cc:260] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.475237 23776 raft_consensus.cc:3060] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:39.474346 23927 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:39.478842 23856 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:39.480295 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:39.481186 23779 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:39.481632 23779 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:39.478463 23824 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:39.481725 23926 raft_consensus.cc:1240] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:39.484927 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:39.485968 25168 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:39.482653 23779 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:39.490000 23779 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:39.502326 23776 raft_consensus.cc:2468] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:39.504233 25168 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:39.505556 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:39.504228 23779 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:39.511125 23926 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.509019 29866 raft_consensus.cc:2804] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:39.510445 29822 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:39.538096 29822 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:39.539220 29822 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:39.543202 29866 raft_consensus.cc:697] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:39.531284 29904 ts_tablet_manager.cc:933] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:15:39.561921 29898 log.cc:927] Time spent T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 12:15:39.544221 29866 consensus_queue.cc:237] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:39.576889 29854 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.581219 23856 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.614406 29904 tablet_copy_client.cc:323] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
W20260709 12:15:39.607919 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 130309us
I20260709 12:15:39.704206 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet 1b93ab9c24064567b717af205bc2ed2a from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:15:39.704952 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet 1b93ab9c24064567b717af205bc2ed2a from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-1b93ab9c24064567b717af205bc2ed2a
W20260709 12:15:39.752048 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 73418us
W20260709 12:15:39.755164 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 158665us
I20260709 12:15:39.784168 23795 tablet_copy_source_session.cc:215] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:39.787864 29904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b93ab9c24064567b717af205bc2ed2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:39.804883 23679 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 14 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:39.806859 23679 catalog_manager.cc:5697] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 13 to 14, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:39.812083 23928 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:39.822712 23679 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.825222 29876 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:15:39.871510 23898 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ed3fc49956b40aa86095410b7c7e5fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:39.884204 23859 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:39.887830 29838 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:39.893328 23932 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:39.906394 23856 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:39.908936 29872 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:39.919044 23777 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:39.931273 23933 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:39.881495 29876 tablet_bootstrap.cc:492] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:39.941825 29839 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:39.947695 29876 ts_tablet_manager.cc:1403] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.859s	user 0.043s	sys 0.038s
I20260709 12:15:39.954025 29801 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:15:39.967527 29876 raft_consensus.cc:359] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:39.972620 29876 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:39.973424 29876 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:39.979539 29876 ts_tablet_manager.cc:1434] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
W20260709 12:15:40.062697 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82311.6us
I20260709 12:15:40.063784 29926 ts_tablet_manager.cc:933] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:40.077901 23677 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:40.084509 25170 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:40.084645 23932 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:40.088979 29858 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:40.091223 29858 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:40.101970 29857 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:40.114014 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c6f2d1f3ed104c1e99f82ebe04843248 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:15:40.114627 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c6f2d1f3ed104c1e99f82ebe04843248 on tablet c6f2d1f3ed104c1e99f82ebe04843248 with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:40.117920 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 140545us
I20260709 12:15:40.127310 23859 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:40.161645 23776 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:40.161046 23779 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:40.164738 23777 raft_consensus.cc:1275] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:40.193702 29858 consensus_queue.cc:1048] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:40.188515 29904 tablet_copy_client.cc:806] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:40.196163 29904 tablet_copy_client.cc:670] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:40.200465 29926 tablet_copy_client.cc:323] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:15:40.207782 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet 5038ac294ceb4758b5b5de70f2469d5d from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:15:40.208472 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet 5038ac294ceb4758b5b5de70f2469d5d from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-5038ac294ceb4758b5b5de70f2469d5d
I20260709 12:15:40.213353 29904 tablet_copy_client.cc:538] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:40.224762 29866 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:40.243186 25170 raft_consensus.cc:1217] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.8->[11.9-11.9]   Dedup: 11.9->[]
I20260709 12:15:40.296783 23677 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:40.303179 29876 ts_tablet_manager.cc:933] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:40.313577 23795 tablet_copy_source_session.cc:215] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:40.317667 29926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5038ac294ceb4758b5b5de70f2469d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:40.344952 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.389365 29876 tablet_copy_client.cc:323] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:15:40.446831 29904 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:40.476094 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 5ed3fc49956b40aa86095410b7c7e5fa from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:15:40.476773 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 5ed3fc49956b40aa86095410b7c7e5fa from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-5ed3fc49956b40aa86095410b7c7e5fa
I20260709 12:15:40.515632 23944 tablet_copy_source_session.cc:215] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:40.531865 29876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed3fc49956b40aa86095410b7c7e5fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:40.563176 29926 tablet_copy_client.cc:806] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:40.579275 29926 tablet_copy_client.cc:670] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:40.584357 29926 tablet_copy_client.cc:538] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:40.598807 29876 tablet_copy_client.cc:806] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:40.604877 29876 tablet_copy_client.cc:670] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:40.614786 29876 tablet_copy_client.cc:538] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:40.644214 29926 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:40.663834 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:40.664525 23856 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:40.670617 23778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:40.671257 23778 raft_consensus.cc:606] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:40.672545 29876 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:40.676107 29917 raft_consensus.cc:993] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:40.676703 29866 raft_consensus.cc:1081] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:40.676844 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:40.677424 23856 raft_consensus.cc:606] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:40.678416 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:40.682849 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:15:40.684893 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:40.685222 23779 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:40.685757 23779 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.694202 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.711383 23779 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:40.714345 23779 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 21 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:40.714529 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:40.715080 23928 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.716006 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:40.716590 23856 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:40.716882 23856 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:40.717618 23856 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:40.718683 23856 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:40.730010 23856 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 21.
I20260709 12:15:40.731199 23751 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:40.746304 23928 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 21.
I20260709 12:15:40.735916 29843 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:40.752832 29843 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:40.758764 29843 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:40.860577 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 56274us
I20260709 12:15:40.919780 29904 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:40.921949 29904 tablet_bootstrap.cc:492] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:40.941033 29857 raft_consensus.cc:1064] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:40.961474 29876 tablet_bootstrap.cc:700] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Time spent opening tablet: real 0.174s	user 0.010s	sys 0.001s
I20260709 12:15:40.934758 29904 ts_tablet_manager.cc:1403] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.488s	user 0.048s	sys 0.008s
I20260709 12:15:40.950954 29857 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:41.104070 23681 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 20 to 21, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 21 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:41.366802 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 152033us
W20260709 12:15:41.387684 29926 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:15:41.429473 29862 raft_consensus.cc:993] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:41.430321 29862 raft_consensus.cc:1081] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
W20260709 12:15:41.438519 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 162834us
I20260709 12:15:41.453356 29904 raft_consensus.cc:359] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.454771 29904 raft_consensus.cc:740] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
W20260709 12:15:41.469287 29845 log.cc:927] Time spent T 00000000000000000000000000000000 P 8f6194fc84354daa9cd0b57de46d8e32: Append to log took a long time: real 0.123s	user 0.001s	sys 0.001s
I20260709 12:15:41.471619 29904 consensus_queue.cc:260] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.474975 29904 ts_tablet_manager.cc:1434] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.500s	user 0.007s	sys 0.006s
I20260709 12:15:41.476373 29962 raft_consensus.cc:993] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:41.477202 29801 raft_consensus.cc:1081] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
W20260709 12:15:41.480100 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 241600us
W20260709 12:15:41.544709 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 219438us
W20260709 12:15:41.475423 29904 ts_tablet_manager.cc:1467] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Tablet startup took 504ms
I20260709 12:15:41.585377 23857 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:41.591473 29956 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:41.607800 23777 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:15:41.610736 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 173359us
W20260709 12:15:41.614238 29926 log.cc:927] Time spent T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:15:41.657470 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:41.658334 23779 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:41.658767 23779 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.660175 25170 raft_consensus.cc:1217] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
W20260709 12:15:41.555989 29904 ts_tablet_manager.cc:1469] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Trace:
0709 12:15:39.466810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.466956 (+   146us) service_pool.cc:224] Handling call
0709 12:15:39.551772 (+ 84816us) ts_tablet_manager.cc:934] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
0709 12:15:39.786580 (+234808us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:15:39.804430 (+ 17850us) tablet_metadata.cc:665] Metadata flushed
0709 12:15:39.804555 (+   125us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:15:39.967049 (+162494us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:15:39.967381 (+   332us) tablet_replica.cc:206] Creating consensus instance
0709 12:15:40.154068 (+186687us) ts_tablet_manager.cc:989] Replica registered.
0709 12:15:40.158163 (+  4095us) inbound_call.cc:177] Queueing success response
0709 12:15:40.446398 (+288235us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:15:40.979706 (+533308us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:15:40.981152 (+  1446us) tablet_replica.cc:261] Starting instrumentation
0709 12:15:40.982859 (+  1707us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":355,"mutex_wait_us":6207,"raft.queue_time_us":41643,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":173,"tablet-copy.queue_time_us":63696,"tablet-download-pool.queue_time_us":123,"tablet-download-pool.run_cpu_time_us":1084,"tablet-download-pool.run_wall_time_us":12390,"thread_start_us":81448,"threads_started":2}
I20260709 12:15:41.689952 23855 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:41.712157 23928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:41.712759 23928 raft_consensus.cc:493] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:41.713173 23928 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:41.733822 29920 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:41.735035 29876 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:41.762792 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-1b93ab9c24064567b717af205bc2ed2a received from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:41.763461 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-1b93ab9c24064567b717af205bc2ed2a on tablet 1b93ab9c24064567b717af205bc2ed2a with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:41.764057 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52570.9us
I20260709 12:15:41.777938 23779 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:41.787760 29876 tablet_bootstrap.cc:492] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:41.799763 29926 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:41.800493 23932 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:15:41.801075 29926 tablet_bootstrap.cc:492] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:41.802068 29926 ts_tablet_manager.cc:1403] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 1.158s	user 0.092s	sys 0.026s
I20260709 12:15:41.810915 29955 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:41.812040 29876 ts_tablet_manager.cc:1403] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 1.140s	user 0.047s	sys 0.037s
I20260709 12:15:41.837430 29876 raft_consensus.cc:359] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.841014 29876 raft_consensus.cc:740] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:41.841718 29876 consensus_queue.cc:260] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.808991 23928 raft_consensus.cc:515] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:41.863037 29876 ts_tablet_manager.cc:1434] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.002s
I20260709 12:15:41.867561 23926 raft_consensus.cc:1240] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:41.871029 25168 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:41.883518 23928 leader_election.cc:290] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:41.887652 25170 raft_consensus.cc:1217] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 12:15:41.891530 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:41.897032 23783 raft_consensus.cc:3055] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:41.897360 23783 raft_consensus.cc:740] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:41.898180 23783 consensus_queue.cc:260] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:41.919605 23783 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:41.920261 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-5ed3fc49956b40aa86095410b7c7e5fa received from {username='slave'} at 127.0.0.1:55680
I20260709 12:15:41.924028 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-5ed3fc49956b40aa86095410b7c7e5fa on tablet 5ed3fc49956b40aa86095410b7c7e5fa with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:41.932930 29857 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:41.942133 29926 raft_consensus.cc:359] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.943181 29926 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:41.943059 29968 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:41.943956 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:41.944582 23856 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:41.944852 23856 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:41.945473 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9d6097cb8d40efa55ff97de5b26130" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:41.946075 25170 raft_consensus.cc:3060] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:41.945721 23856 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:41.947968 23856 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.946583 23779 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:41.944044 29926 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:41.979671 25170 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:41.983685 23783 raft_consensus.cc:2468] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:15:41.992290 29926 ts_tablet_manager.cc:1434] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.190s	user 0.009s	sys 0.000s
I20260709 12:15:41.992489 25174 raft_consensus.cc:1217] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 12:15:41.995661 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-5038ac294ceb4758b5b5de70f2469d5d received from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:41.996155 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-5038ac294ceb4758b5b5de70f2469d5d on tablet 5038ac294ceb4758b5b5de70f2469d5d with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:42.008745 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:42.009451 23927 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:42.019199 23856 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
W20260709 12:15:42.027779 29981 log.cc:927] Time spent T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:15:42.020021 23899 leader_election.cc:304] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:42.063871 23751 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:42.073715 29756 raft_consensus.cc:2804] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:42.079914 29756 raft_consensus.cc:697] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:42.080699 23927 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 14.
I20260709 12:15:42.081056 29756 consensus_queue.cc:237] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:42.088671 29801 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:42.089326 29801 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:42.090309 29801 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:42.109825 29716 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.111928 25174 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:42.168843 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 99947.6us
W20260709 12:15:42.190011 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 126220us
I20260709 12:15:42.196199 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:42.287171 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 47617.5us
W20260709 12:15:42.288932 29989 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 12:15:42.293001 29983 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:15:42.308234 23783 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.326093 29920 raft_consensus.cc:1064] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:15:42.330054 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 54548.1us
I20260709 12:15:42.332146 29920 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:42.350309 23681 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 13 to 14, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.416050 23856 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.441514 29839 raft_consensus.cc:1064] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:42.481601 25174 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:42.486454 23851 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:42.493294 23930 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:42.500963 29872 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:15:42.503582 29964 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:42.523128 23677 catalog_manager.cc:5697] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 17 to 18, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 18 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.512051 29920 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:15:42.567906 29839 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:42.602181 25168 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:42.603928 29992 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:42.612787 23780 raft_consensus.cc:1275] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:42.614924 29944 raft_consensus.cc:1064] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:42.618672 29944 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:42.620600 29992 consensus_queue.cc:1048] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:42.626996 23932 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:42.641129 23857 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:42.642324 23856 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:42.643790 25170 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:42.645674 29992 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:42.646908 29939 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:42.645558 29854 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:15:42.668584 29993 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.671310 25172 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.682782 23857 raft_consensus.cc:2955] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.688110 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c6f2d1f3ed104c1e99f82ebe04843248 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:42.711881 23851 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.713064 25174 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.713629 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 10 to 11, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 11 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:42.715044 29839 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:42.716095 29756 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:42.814079 23783 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.819625 29839 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:42.831137 23860 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.832839 29920 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:42.840418 25172 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.807482 29987 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.845914 29839 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.855618 23930 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:42.870644 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet c6f2d1f3ed104c1e99f82ebe04843248 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:42.873546 23932 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.883275 25174 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.901396 23928 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:42.876709 25170 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:42.900641 29854 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:15:42.947801 29964 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:15:43.063843 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 71542.3us
I20260709 12:15:43.023742 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:43.069551 30011 tablet_replica.cc:333] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:15:43.084374 23860 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.118696 30011 raft_consensus.cc:2243] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.044363 29854 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:15:43.119760 30011 raft_consensus.cc:2272] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:43.150941 30006 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.105s	user 0.000s	sys 0.006s
I20260709 12:15:43.153158 23677 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.162037 30011 ts_tablet_manager.cc:1916] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:15:43.193504 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 120454us
W20260709 12:15:43.196167 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 112733us
W20260709 12:15:43.260154 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 175394us
I20260709 12:15:43.282450 23677 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.319806 29964 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.339114 23776 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.376781 25175 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:43.434051 30022 log.cc:927] Time spent T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.241s	user 0.000s	sys 0.002s
W20260709 12:15:43.496910 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 270582us
W20260709 12:15:43.527139 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 248291us
I20260709 12:15:43.542414 23857 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.541993 23683 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.576759 23932 consensus_queue.cc:237] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:43.601033 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.608084 25175 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:43.613078 23780 raft_consensus.cc:1275] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:15:43.624588 23897 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.630292 30011 ts_tablet_manager.cc:1929] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.10
I20260709 12:15:43.630913 30011 log.cc:1199] T c6f2d1f3ed104c1e99f82ebe04843248 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/c6f2d1f3ed104c1e99f82ebe04843248
I20260709 12:15:43.631542 23780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:43.624277 30044 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:43.635461 23780 raft_consensus.cc:606] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:43.657378 30040 consensus_queue.cc:1048] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:43.673794 23783 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:43.676961 23851 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.677801 23933 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.711292 23930 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:43.713277 23930 raft_consensus.cc:606] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:43.723894 23925 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:43.724725 29920 raft_consensus.cc:993] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:43.729761 29920 raft_consensus.cc:1081] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:43.729444 29944 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:43.732086 30018 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.739704 23860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:43.740343 23860 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.740694 23860 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.750308 25174 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:43.750005 29920 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.757652 30038 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:43.758869 30040 raft_consensus.cc:993] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:43.759361 30040 raft_consensus.cc:1081] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:43.758381 23933 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.765090 23780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:43.765743 23780 raft_consensus.cc:606] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:43.766541 23779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:15:43.767434 23779 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:43.767818 23779 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.761108 25175 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.775648 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:15:43.748456 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet c6f2d1f3ed104c1e99f82ebe04843248 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:43.782737 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:43.801241 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:43.805447 23779 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.806711 23860 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.808403 23783 raft_consensus.cc:2955] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.815256 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:43.816008 23851 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.826180 23779 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:43.828514 25170 raft_consensus.cc:2955] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.840155 23928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:43.840807 23928 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.837294 23778 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:43.852434 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:43.853116 23933 raft_consensus.cc:3055] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:43.853428 23933 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:43.839136 23860 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:43.856498 23933 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.861976 23928 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:43.863487 23824 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:43.870371 23933 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.894852 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:43.895650 23783 raft_consensus.cc:3055] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:43.895952 23783 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:43.870780 30026 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:43.905160 30026 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:43.914003 23851 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:43.925890 30026 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.914321 23933 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:43.939008 25170 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:43.943197 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b93ab9c24064567b717af205bc2ed2a with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:43.938629 23751 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:43.952785 29920 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:43.954236 29920 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:43.955408 29920 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.975937 23858 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:43.980789 29987 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:43.985292 29984 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:43.990998 23783 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:43.994496 23783 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.995937 23928 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:43.995693 29984 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:43.997675 23858 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:44.007825 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ed3fc49956b40aa86095410b7c7e5fa with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:44.020423 25170 raft_consensus.cc:2955] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:44.027920 23857 raft_consensus.cc:1240] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:44.057310 23783 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:15:44.098062 29944 raft_consensus.cc:993] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:44.099031 29944 raft_consensus.cc:1081] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:44.101045 23857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:44.101645 23857 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:44.102057 23857 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:44.101675 23683 catalog_manager.cc:5697] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:44.105937 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5038ac294ceb4758b5b5de70f2469d5d with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:44.140913 23683 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 15 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:44.161446 23669 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 5038ac294ceb4758b5b5de70f2469d5d with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 11 but the committed config has opid_index of 12
I20260709 12:15:44.166522 23857 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:44.183068 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:44.183730 23783 raft_consensus.cc:3055] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:44.183764 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71cb2ce294574daa8d7244ae252e8aa6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:44.183863 23857 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:44.184401 23783 raft_consensus.cc:740] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:44.184595 25170 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:44.192215 23783 consensus_queue.cc:260] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:44.193468 23783 raft_consensus.cc:3060] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:44.196861 25170 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:44.202684 23824 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:44.203773 30048 raft_consensus.cc:2804] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:44.204334 30048 raft_consensus.cc:697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:44.205322 30048 consensus_queue.cc:237] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:44.193857 23677 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 14 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.209347 23677 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 11 to 12, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:44.239490 23783 raft_consensus.cc:2468] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:44.344310 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ed3fc49956b40aa86095410b7c7e5fa with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:35466
I20260709 12:15:44.370002 23928 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:44.370368 30059 tablet_replica.cc:333] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:15:44.372166 29944 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:44.381299 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 5038ac294ceb4758b5b5de70f2469d5d with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:50860
I20260709 12:15:44.383827 30061 tablet_replica.cc:333] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:15:44.385134 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b93ab9c24064567b717af205bc2ed2a with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:35466
I20260709 12:15:44.390300 30059 raft_consensus.cc:2243] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:44.391043 30059 raft_consensus.cc:2272] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:44.390441 23681 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.391729 23677 catalog_manager.cc:5697] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:44.425592 30061 raft_consensus.cc:2243] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:44.426282 30061 raft_consensus.cc:2272] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:44.432228 30061 ts_tablet_manager.cc:1916] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:44.440655 30059 ts_tablet_manager.cc:1916] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:44.441243 23783 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:44.443513 30026 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:44.447677 23857 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:44.449314 29920 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:44.460070 23933 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:44.461954 30026 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:44.544970 30059 ts_tablet_manager.cc:1929] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 12:15:44.546169 30059 log.cc:1199] T 5ed3fc49956b40aa86095410b7c7e5fa P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/5ed3fc49956b40aa86095410b7c7e5fa
I20260709 12:15:44.550324 30061 ts_tablet_manager.cc:1929] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 12:15:44.556900 30059 tablet_replica.cc:333] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
W20260709 12:15:44.577798 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 57287.4us
I20260709 12:15:44.585206 30061 log.cc:1199] T 5038ac294ceb4758b5b5de70f2469d5d P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/5038ac294ceb4758b5b5de70f2469d5d
I20260709 12:15:44.612442 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 5ed3fc49956b40aa86095410b7c7e5fa (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:44.639480 30059 raft_consensus.cc:2243] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:44.641113 30059 raft_consensus.cc:2272] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:44.644152 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46756.4us
I20260709 12:15:44.651528 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 5038ac294ceb4758b5b5de70f2469d5d (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:44.661692 30059 ts_tablet_manager.cc:1916] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:44.729632 23783 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:15:44.732887 30068 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:44.754415 25170 raft_consensus.cc:1275] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:15:44.757807 30068 consensus_queue.cc:1048] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:44.798262 30059 ts_tablet_manager.cc:1929] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:15:44.803877 30059 log.cc:1199] T 1b93ab9c24064567b717af205bc2ed2a P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/1b93ab9c24064567b717af205bc2ed2a
I20260709 12:15:44.806056 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 1b93ab9c24064567b717af205bc2ed2a (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:45.248386 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:45.400719 23851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:45.401378 23851 raft_consensus.cc:606] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:45.424330 23783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:45.424945 23783 raft_consensus.cc:606] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:45.427862 23851 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:45.428493 23851 raft_consensus.cc:606] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:45.432123 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:15:45.435321 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:45.438433 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:45.724117 30073 raft_consensus.cc:993] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:45.724640 30073 raft_consensus.cc:1081] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:45.726487 23926 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45620
I20260709 12:15:45.727073 23926 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.727442 23926 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:45.748390 23926 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:45.751544 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:45.752260 23851 raft_consensus.cc:3055] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:45.752543 23851 raft_consensus.cc:740] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:45.753283 23851 consensus_queue.cc:260] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:45.754451 23851 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:45.754773 23858 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:45.781199 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1567865ea02a4884996ae9d8beb9fa29" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:45.781821 25172 raft_consensus.cc:3060] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:45.791705 25172 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:15:45.794203 23898 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:45.795408 30017 raft_consensus.cc:2804] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:45.795960 30017 raft_consensus.cc:697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:45.796854 30017 consensus_queue.cc:237] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:45.792266 23926 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:45.821033 23851 raft_consensus.cc:2468] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
W20260709 12:15:45.829401 23824 consensus_peers.cc:597] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98e5bc86804f4f979f58779caa696814. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:45.833652 23783 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:45.836314 30074 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:15:45.853348 23677 catalog_manager.cc:5697] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 13 to 14, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 14 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.861303 23926 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:45.889834 30073 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:45.901079 30074 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:45.913906 23783 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
W20260709 12:15:45.916393 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47802.9us
I20260709 12:15:45.923060 23851 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:45.933508 30084 raft_consensus.cc:993] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:15:45.934079 30084 raft_consensus.cc:1081] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:15:45.934688 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:45.949198 25170 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:53494
I20260709 12:15:45.949795 25170 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:45.950120 25170 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:45.950266 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 49877us
I20260709 12:15:45.965744 25170 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:46.000423 30073 raft_consensus.cc:993] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:46.001062 30088 raft_consensus.cc:1081] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:46.003176 23783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:46.003798 23783 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.004137 23783 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.012578 23783 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:46.021721 23783 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:46.019972 23926 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.022821 25170 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:46.022782 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:46.023636 23933 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.025012 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:46.025811 23860 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.034045 23933 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:46.025327 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:46.035256 23783 raft_consensus.cc:3055] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:46.035552 23783 raft_consensus.cc:740] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:46.036638 23860 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:15:46.038188 23749 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:46.038966 30071 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:46.039492 30071 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:46.040859 25119 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:46.041741 29707 raft_consensus.cc:2804] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:46.042220 29707 raft_consensus.cc:697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:46.043059 29707 consensus_queue.cc:237] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:46.048305 23933 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:46.050382 30071 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:46.052307 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:46.053049 23860 raft_consensus.cc:3055] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:46.053025 23783 consensus_queue.cc:260] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:46.053615 23860 raft_consensus.cc:740] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
W20260709 12:15:46.054597 23824 consensus_peers.cc:597] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 053977846bfa4adf9703ea669a81f41d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:46.068984 23783 raft_consensus.cc:3060] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.077267 23860 consensus_queue.cc:260] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:46.093969 23779 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:46.097324 30073 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:46.108537 23783 raft_consensus.cc:2468] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:15:46.116333 23860 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.143554 23860 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:46.182279 23677 catalog_manager.cc:5697] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 17 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:46.183679 30094 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:46.192237 23679 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 16 to 17, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 17 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:46.200259 25170 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:15:46.201876 30040 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:46.218873 30073 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.225081 23778 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.253898 23679 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.318109 23933 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.330924 23860 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:15:46.414073 23824 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc09548ba8eb4f01bf6d74a33d072dda. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:46.415505 23933 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:15:46.422746 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 120575us
I20260709 12:15:46.432025 23783 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:15:46.437240 30073 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:46.439416 30073 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:15:46.497660 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 65915.9us
I20260709 12:15:46.500036 23860 raft_consensus.cc:1275] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:15:46.503870 30088 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.505208 23783 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.512223 30107 consensus_queue.cc:1048] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:46.538776 30111 ts_tablet_manager.cc:933] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:46.554087 23926 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:46.561702 23683 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.571069 30117 ts_tablet_manager.cc:933] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:46.572980 30106 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:46.599475 23783 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:15:46.602077 29707 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:46.611565 23860 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:46.613080 30104 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:46.618848 30117 tablet_copy_client.cc:323] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:15:46.622546 23933 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.624828 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 98e5bc86804f4f979f58779caa696814 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:15:46.625506 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 98e5bc86804f4f979f58779caa696814 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-98e5bc86804f4f979f58779caa696814
I20260709 12:15:46.644305 23857 raft_consensus.cc:1275] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 17. (index mismatch)
I20260709 12:15:46.660255 30087 consensus_queue.cc:1048] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:15:46.667083 23928 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:46.678980 30111 tablet_copy_client.cc:323] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:15:46.681093 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:46.687752 23869 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 053977846bfa4adf9703ea669a81f41d from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:15:46.688508 23869 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 053977846bfa4adf9703ea669a81f41d from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-053977846bfa4adf9703ea669a81f41d
I20260709 12:15:46.704689 23869 tablet_copy_source_session.cc:215] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:46.710410 30111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053977846bfa4adf9703ea669a81f41d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:46.712409 23870 tablet_copy_source_session.cc:215] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:46.743431 30117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98e5bc86804f4f979f58779caa696814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:46.769450 23858 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:46.772372 23779 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:46.784089 23780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:46.784790 23780 raft_consensus.cc:606] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:46.791244 23928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:46.792006 23928 raft_consensus.cc:606] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:46.793656 30116 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:46.794564 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:46.794664 30115 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:46.796949 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:46.797797 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:46.824312 23898 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1031da3c683416ebfc283d98b45867d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:46.870693 30111 tablet_copy_client.cc:806] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:46.871443 30111 tablet_copy_client.cc:670] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:46.906157 23858 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.931751 30111 tablet_copy_client.cc:538] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:46.939703 30117 tablet_copy_client.cc:806] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:46.940506 30117 tablet_copy_client.cc:670] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:46.931262 30115 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:46.988300 30084 raft_consensus.cc:993] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:46.988816 30084 raft_consensus.cc:1081] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:46.991456 23857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:46.992161 23857 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:46.992534 23857 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:46.998620 30117 tablet_copy_client.cc:538] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:47.013281 30117 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:47.058879 30133 ts_tablet_manager.cc:933] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:47.058367 23679 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:47.116345 30111 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:47.128213 23780 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:47.138051 23857 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:47.156453 25174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:47.157130 25174 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:47.155381 23857 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:47.165844 25174 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:47.167778 23824 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:47.168852 30073 raft_consensus.cc:2804] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:47.169399 30073 raft_consensus.cc:697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:47.170324 30073 consensus_queue.cc:237] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:47.192121 30133 tablet_copy_client.cc:323] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:15:47.194454 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet fc09548ba8eb4f01bf6d74a33d072dda from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:15:47.195132 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet fc09548ba8eb4f01bf6d74a33d072dda from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-fc09548ba8eb4f01bf6d74a33d072dda
I20260709 12:15:47.196098 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:47.196729 23780 raft_consensus.cc:3055] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:47.197016 23780 raft_consensus.cc:740] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:47.197814 23780 consensus_queue.cc:260] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:47.199106 23780 raft_consensus.cc:3060] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:47.223421 23870 tablet_copy_source_session.cc:215] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:47.234261 30017 raft_consensus.cc:993] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:47.236234 30116 raft_consensus.cc:1081] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:47.237969 30137 ts_tablet_manager.cc:933] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:47.238178 23783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2844521776744308787188de9041ebd"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43438
I20260709 12:15:47.244061 23783 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:47.244441 23783 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.245217 30133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc09548ba8eb4f01bf6d74a33d072dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:47.265478 30133 tablet_copy_client.cc:806] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:47.263648 23677 catalog_manager.cc:5697] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 13 to 14, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 14 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.266199 30133 tablet_copy_client.cc:670] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:47.263286 23780 raft_consensus.cc:2468] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:15:47.270373 30133 tablet_copy_client.cc:538] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:47.274240 30117 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:47.281338 30133 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:47.293895 30117 tablet_bootstrap.cc:492] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:47.288148 23783 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:47.306007 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:47.306490 23783 leader_election.cc:290] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:47.306674 23851 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.311342 30117 ts_tablet_manager.cc:1403] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.298s	user 0.043s	sys 0.031s
I20260709 12:15:47.316570 23851 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:47.326678 23751 leader_election.cc:304] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:47.330312 30106 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:47.330964 30106 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:47.331923 30106 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:47.322633 30117 raft_consensus.cc:359] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.324934 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:47.342682 23926 raft_consensus.cc:3055] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:47.343020 23926 raft_consensus.cc:740] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:47.343828 23926 consensus_queue.cc:260] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:47.347169 23926 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.349543 30137 tablet_copy_client.cc:323] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:15:47.351567 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet e1031da3c683416ebfc283d98b45867d from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:15:47.352210 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet e1031da3c683416ebfc283d98b45867d from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-e1031da3c683416ebfc283d98b45867d
I20260709 12:15:47.372879 23926 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 17.
I20260709 12:15:47.346405 30117 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:47.381163 30117 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.388968 30117 ts_tablet_manager.cc:1434] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.074s	user 0.002s	sys 0.004s
I20260709 12:15:47.390957 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-98e5bc86804f4f979f58779caa696814 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:15:47.391456 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-98e5bc86804f4f979f58779caa696814 on tablet 98e5bc86804f4f979f58779caa696814 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:47.400446 23944 tablet_copy_source_session.cc:215] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:47.404945 30137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1031da3c683416ebfc283d98b45867d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:47.432237 30111 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:47.433383 30111 tablet_bootstrap.cc:492] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:47.436322 30111 ts_tablet_manager.cc:1403] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.320s	user 0.044s	sys 0.000s
I20260709 12:15:47.450634 23677 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 16 to 17, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 17 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:47.455961 30111 raft_consensus.cc:359] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.456975 30111 raft_consensus.cc:740] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:47.457693 30111 consensus_queue.cc:260] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.464787 30137 tablet_copy_client.cc:806] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:47.471848 30137 tablet_copy_client.cc:670] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:47.475224 30111 ts_tablet_manager.cc:1434] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:15:47.489722 30133 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:47.496948 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-053977846bfa4adf9703ea669a81f41d received from {username='slave'} at 127.0.0.1:46448
I20260709 12:15:47.504696 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-053977846bfa4adf9703ea669a81f41d on tablet 053977846bfa4adf9703ea669a81f41d with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:47.497277 30137 tablet_copy_client.cc:538] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:47.515713 30133 tablet_bootstrap.cc:492] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:47.516613 30133 ts_tablet_manager.cc:1403] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.236s	user 0.042s	sys 0.007s
I20260709 12:15:47.521981 30133 raft_consensus.cc:359] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.535791 30133 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:47.518450 30137 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:47.537827 25174 raft_consensus.cc:1217] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.9->[11.10-11.10]   Dedup: 11.10->[]
I20260709 12:15:47.544091 30133 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:47.550357 30133 ts_tablet_manager.cc:1434] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:15:47.552932 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-fc09548ba8eb4f01bf6d74a33d072dda received from {username='slave'} at 127.0.0.1:46448
I20260709 12:15:47.553399 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-fc09548ba8eb4f01bf6d74a33d072dda on tablet fc09548ba8eb4f01bf6d74a33d072dda with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:47.592756 25174 raft_consensus.cc:1217] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 12:15:47.601890 23783 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:47.714737 25174 raft_consensus.cc:1275] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:47.734560 30073 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:47.782720 25175 raft_consensus.cc:1217] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 12:15:47.991906 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:48.094992 30137 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:48.097504 30137 tablet_bootstrap.cc:492] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:48.098500 30137 ts_tablet_manager.cc:1403] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.580s	user 0.053s	sys 0.072s
I20260709 12:15:48.101969 30137 raft_consensus.cc:359] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:48.103012 30137 raft_consensus.cc:740] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:48.103907 30137 consensus_queue.cc:260] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:48.105944 30137 ts_tablet_manager.cc:1434] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:15:48.191223 23783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
W20260709 12:15:48.201133 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56253.2us
I20260709 12:15:48.191913 23783 raft_consensus.cc:606] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:48.209000 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 59571.8us
I20260709 12:15:48.215459 23860 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:48.210559 30148 consensus_queue.cc:1048] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:15:48.253500 30146 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:48.256963 23860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:48.257754 23860 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:48.263185 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:48.265144 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:48.265950 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:48.308387 23926 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:48.336503 25170 raft_consensus.cc:1217] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.12->[17.13-17.13]   Dedup: 17.13->[]
W20260709 12:15:48.403220 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 50264.2us
I20260709 12:15:48.407569 30160 raft_consensus.cc:1064] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:15:48.407769 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 139151us
I20260709 12:15:48.408209 30158 raft_consensus.cc:1064] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:48.411428 30158 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:48.414763 30160 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:48.468753 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 115479us
I20260709 12:15:48.475296 30148 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:48.476011 30148 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:48.516711 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-e1031da3c683416ebfc283d98b45867d received from {username='slave'} at 127.0.0.1:55680
I20260709 12:15:48.517280 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-e1031da3c683416ebfc283d98b45867d on tablet e1031da3c683416ebfc283d98b45867d with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:48.537258 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 133124us
I20260709 12:15:48.555092 23932 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:48.558241 30073 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:48.574532 25172 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:48.620424 23927 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:48.621235 25173 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:15:48.625948 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 223230us
I20260709 12:15:48.627151 30150 raft_consensus.cc:1064] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:48.627732 30147 raft_consensus.cc:993] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:48.628223 30147 raft_consensus.cc:1081] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:48.629667 30150 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:48.636435 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 230749us
I20260709 12:15:48.656032 30103 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:48.670810 30160 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:48.674011 30160 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:48.680783 23933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45606
I20260709 12:15:48.681432 23933 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:48.681792 23933 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.690706 23933 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:48.678920 25174 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:48.714046 23778 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:48.714485 23777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:48.715034 23777 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:48.715394 23777 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.717031 23783 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:48.720499 30155 raft_consensus.cc:1064] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:48.728518 30150 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:48.735846 23933 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:48.770510 23777 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:48.781450 23779 raft_consensus.cc:1240] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:48.786984 23777 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:48.819257 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 57827us
I20260709 12:15:48.789937 30169 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:15:48.842954 23779 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:48.863052 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:48.864336 25175 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.868002 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:48.891942 23851 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:48.895799 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:48.895555 30097 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:15:48.902132 25175 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:48.902614 23854 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:48.907302 30182 consensus_queue.cc:1059] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:48.909013 30017 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:48.893671 30148 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:15:48.903072 23854 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:48.911234 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:48.911896 23860 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.916141 30094 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:48.834177 30155 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:48.922132 25170 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:48.925447 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 48535.2us
I20260709 12:15:48.935822 23860 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:48.947237 23900 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:48.949329 30112 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:48.952086 23779 raft_consensus.cc:3055] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:48.952610 23779 raft_consensus.cc:740] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:48.954413 30150 raft_consensus.cc:2804] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:48.954645 23779 consensus_queue.cc:260] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:48.956000 23858 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:48.956362 23779 raft_consensus.cc:3060] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.953068 23854 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:48.972344 30145 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:48.973026 30145 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:48.976538 30145 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:48.981241 23854 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.983429 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:48.987361 30150 raft_consensus.cc:697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:15:49.014431 23806 debug-util.cc:398] Leaking SignalData structure 0x7b0800372200 after lost signal to thread 22632
I20260709 12:15:49.114624 23779 raft_consensus.cc:2468] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
W20260709 12:15:49.115823 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08003699c0 after lost signal to thread 23717
W20260709 12:15:49.117185 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08000cc2c0 after lost signal to thread 23809
W20260709 12:15:49.118395 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08000de480 after lost signal to thread 23884
I20260709 12:15:49.118477 23854 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 16.
I20260709 12:15:49.120440 30150 consensus_queue.cc:237] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:15:49.123884 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 127808us
W20260709 12:15:49.124531 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 128748us
W20260709 12:15:49.128412 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 131580us
W20260709 12:15:49.130875 23806 debug-util.cc:398] Leaking SignalData structure 0x7b08003928a0 after lost signal to thread 23958
W20260709 12:15:49.142875 23806 debug-util.cc:398] Leaking SignalData structure 0x7b080021a640 after lost signal to thread 25207
I20260709 12:15:49.150264 23751 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
W20260709 12:15:49.151782 30182 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:15:49.173892 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:49.177871 30164 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:49.229272 23683 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:49.260715 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 92238.8us
W20260709 12:15:49.261955 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 94109.5us
I20260709 12:15:49.265574 23749 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:49.269546 30189 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:15:49.269239 30194 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:49.271889 30192 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:15:49.251178 30164 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
W20260709 12:15:49.280808 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:49.290591 30188 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:49.346136 23933 raft_consensus.cc:1240] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Rejecting Update request from peer 0ee4ab0747914e7abc642312517b5e25 for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:49.430666 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 62306.1us
I20260709 12:15:49.419234 30146 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:49.456851 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 93907.1us
I20260709 12:15:49.439340 30164 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:49.526613 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 108201us
W20260709 12:15:49.744097 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 304098us
W20260709 12:15:49.791329 23749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.799355 23854 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:49.796252 23779 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:49.821393 30203 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:15:49.822850 30194 consensus_meta.cc:287] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 12:15:49.860479 25169 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:49.889779 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:49.851958 30209 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:50.077494 30218 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 12:15:50.160810 23751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.003s	sys 0.010s
W20260709 12:15:50.161304 23751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.003s	sys 0.010s
W20260709 12:15:50.161593 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 703553us
W20260709 12:15:50.191416 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.031s	sys 0.094s
W20260709 12:15:50.191913 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.031s	sys 0.094s
I20260709 12:15:50.198380 23860 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:50.245464 30217 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.246461 30107 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.246152 30217 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:15:50.259142 23824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.002s	sys 0.012s
W20260709 12:15:50.260231 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 514931us
I20260709 12:15:50.261173 30217 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:50.247071 30230 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.266774 30230 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.267927 30228 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.268543 30228 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.269544 30230 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.270936 30228 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.251699 30194 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.274734 30232 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.247128 30107 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.314981 30165 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:50.315270 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.320951 30165 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.331625 30112 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.332404 30112 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.334745 30112 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.289458 30150 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.336421 30150 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.338814 30150 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:50.312644 23860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "526cc198da134bb590358ed73c23d59e"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:50.339908 23860 raft_consensus.cc:606] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.312345 30232 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:50.280167 30194 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.343581 23681 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 15 to 16, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:50.353843 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.354544 25175 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 6.
I20260709 12:15:50.355741 30165 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.362587 23898 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:50.346632 30182 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.363706 30155 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.348165 30220 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.364838 30220 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.367057 30220 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:50.346463 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 85236.8us
I20260709 12:15:50.375398 23780 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:50.378229 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.381387 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.383567 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.392464 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.347263 30094 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.345794 30194 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.364248 30238 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:15:50.396587 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.346881 30073 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.349450 30236 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:50.398221 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.363739 30182 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.401767 23782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.403029 23776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:50.408730 23776 raft_consensus.cc:606] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.410082 23776 raft_consensus.cc:614] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:50.364264 30155 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.403410 30238 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.433640 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:50.435755 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:50.436836 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:50.407619 30239 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.402271 30094 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.414810 30165 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.445036 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.445328 30214 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:15:50.445566 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.447057 30015 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.455411 30107 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:50.463624 30232 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.469980 23931 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 11.
I20260709 12:15:50.405874 30181 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:15:50.436352 30238 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:50.475915 23859 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:50.411381 30073 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.473052 30015 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:50.519543 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 45892.8us
I20260709 12:15:50.523847 30217 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.524271 30226 raft_consensus.cc:493] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:50.472998 30239 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:15:50.528249 23933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.529670 23927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.533715 25170 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:50.535997 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.411865 30236 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:50.542505 23776 raft_consensus.cc:1275] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:50.542778 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d72845b5ad4457bb71a7ad5f5152b4c" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.532701 23928 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.542845 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:50.546801 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.529474 30158 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:15:50.476902 30165 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.571024 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.544615 30104 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:15:50.570326 23780 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.583140 23824 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.593571 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.597194 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.600693 23784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.602313 30094 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.598224 30197 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:50.553490 30217 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.603927 23929 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:50.606181 30217 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.610731 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.615175 30165 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.615602 23934 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.617002 30169 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:50.579216 30177 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:50.553042 30226 raft_consensus.cc:515] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.619133 30239 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:15:50.619947 25174 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.625732 23851 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.625914 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.627061 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.627133 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.626957 23930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.600536 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:50.637387 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 58247.4us
I20260709 12:15:50.659700 30145 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:50.660662 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.661289 30015 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.577416 30158 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.586565 30073 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.663245 30158 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.666420 30226 leader_election.cc:290] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.595191 30104 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.652688 30177 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:50.670192 30177 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:50.671418 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.602710 30236 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:15:50.674135 30237 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.101s	user 0.000s	sys 0.003s
I20260709 12:15:50.666180 30206 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:50.677452 30155 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:50.670043 23782 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.678571 30238 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.679790 23925 raft_consensus.cc:1240] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:50.688256 25169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.688484 30182 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.691691 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.692037 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.693745 23824 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.696558 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.696806 30104 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:50.706938 30155 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:50.710703 30177 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:50.715144 30238 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:50.638171 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 53791.1us
I20260709 12:15:50.717381 23683 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 224) took 1559 ms. Trace:
I20260709 12:15:50.779065 23683 rpcz_store.cc:276] 0709 12:15:49.157723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.157885 (+   162us) service_pool.cc:224] Handling call
0709 12:15:50.717277 (+1559392us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:49.519114 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:49.825821 (+306707us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:49.852096 (+ 26275us) write_op.cc:435] Acquired schema lock
0709 12:15:49.857169 (+  5073us) tablet.cc:662] Decoding operations
0709 12:15:49.924752 (+ 67583us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:49.931345 (+  6593us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:49.933837 (+  2492us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:15:49.988470 (+ 54633us) tablet.cc:701] Row locks acquired
0709 12:15:49.988495 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:15:49.988671 (+   176us) write_op.cc:270] Start()
0709 12:15:49.988790 (+   119us) write_op.cc:276] Timestamp: P: 1783599349988613 usec, L: 0
0709 12:15:49.988812 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:15:49.989597 (+   785us) log.cc:844] Serialized 595 byte log entry
0709 12:15:50.519702 (+530105us) op_driver.cc:464] REPLICATION: finished
0709 12:15:50.686035 (+166333us) write_op.cc:301] APPLY: starting
0709 12:15:50.686123 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:50.698186 (+ 12063us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:50.698289 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:50.705075 (+  6786us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:50.705275 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:15:50.705321 (+    46us) write_op.cc:312] APPLY: finished
0709 12:15:50.705539 (+   218us) log.cc:844] Serialized 25 byte log entry
0709 12:15:50.707925 (+  2386us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:50.708010 (+    85us) write_op.cc:454] Released schema lock
0709 12:15:50.708053 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:50.708822 (+   769us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:50.708853 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":166075,"apply.run_cpu_time_us":4079,"apply.run_wall_time_us":22781,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":211424,"prepare.run_cpu_time_us":14285,"prepare.run_wall_time_us":768938,"raft.queue_time_us":183698,"raft.run_cpu_time_us":35820,"raft.run_wall_time_us":68767,"replication_time_us":530680,"thread_start_us":618927,"threads_started":4,"wal-append.queue_time_us":322111}]]}
I20260709 12:15:50.718669 25170 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:50.721243 23677 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 13 to 14, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:50.724298 23900 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:50.729732 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:50.732136 25168 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:50.734637 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.736476 23825 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:15:50.788851 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 52486.9us
I20260709 12:15:50.738121 23779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.738471 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.738113 30093 consensus_queue.cc:1059] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:50.739101 23778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.739760 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.742925 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.743214 23780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.746445 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1031da3c683416ebfc283d98b45867d" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:50.763098 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:50.785604 23677 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 9 to 10, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:50.804368 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 80140.7us
I20260709 12:15:50.810969 23784 raft_consensus.cc:2393] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 12: Already voted for candidate 0ee4ab0747914e7abc642312517b5e25 in this term.
I20260709 12:15:50.811841 25114 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:50.816792 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:50.820230 30155 raft_consensus.cc:3055] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:15:50.820690 30155 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:50.821606 30155 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.822923 30155 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:50.823915 30154 consensus_queue.cc:1175] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "05679ef5e2b74861aad2a08b6a05e276" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 12:15:50.835388 23776 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:50.815559 23899 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:15:50.837071 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 46046.8us
I20260709 12:15:50.840986 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:50.802744 30097 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.790503 30228 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.855962 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.866233 23900 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:50.871393 23749 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.873761 30232 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:50.844156 30239 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.878240 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:50.881635 30116 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:15:50.883297 30115 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:50.836025 30238 consensus_queue.cc:1048] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:15:50.851164 30165 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:50.899329 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 47069.3us
I20260709 12:15:50.869905 30217 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:50.906230 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 56508.7us
I20260709 12:15:50.862811 30228 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.917824 25113 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:50.918522 25114 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:50.903708 30161 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:15:50.952646 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 80316us
W20260709 12:15:51.027614 30248 log.cc:927] Time spent T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:15:51.009948 30213 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:51.036259 30213 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:50.948989 30161 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.046108 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:51.038549 30213 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:15:51.052237 30252 log.cc:927] Time spent T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.103s	user 0.002s	sys 0.000s
I20260709 12:15:51.068936 30073 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Committing config change with OpId 11.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:51.070945 23931 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:51.124805 30228 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:51.183159 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:51.202957 30212 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:51.211243 30197 raft_consensus.cc:993] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:51.219098 30197 raft_consensus.cc:1081] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:51.221920 23825 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.225279 23929 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "526cc198da134bb590358ed73c23d59e"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45620
I20260709 12:15:51.226573 23929 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:51.227252 23929 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.229467 30151 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:51.200937 30221 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.244257 30221 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:51.248202 30221 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:51.250001 30015 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:51.250952 30015 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.253862 30015 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:51.254558 30015 raft_consensus.cc:2749] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:51.255784 30261 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:51.263414 23825 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.259294 30161 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:51.275300 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:15:51.276144 30265 log.cc:927] Time spent T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:15:51.277472 30197 raft_consensus.cc:2749] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:51.249836 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 331956us
I20260709 12:15:51.297672 25113 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.301868 30145 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:15:51.310500 30177 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:51.302984 30145 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.321168 30145 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:51.310864 23929 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:51.336210 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 56258.8us
I20260709 12:15:51.337970 23929 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:51.338254 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:51.339262 23856 raft_consensus.cc:3055] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:51.339564 23856 raft_consensus.cc:740] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:51.338402 23899 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:51.341981 23856 consensus_queue.cc:260] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.343290 23856 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:51.344143 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46750.5us
I20260709 12:15:51.345230 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.346462 30268 raft_consensus.cc:2749] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.347328 30165 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:51.348052 30213 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:51.350374 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:51.355410 23856 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:51.356886 23900 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:51.358542 30239 raft_consensus.cc:2804] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:51.359186 30239 raft_consensus.cc:697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:51.361989 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:51.364229 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:51.360492 30239 consensus_queue.cc:237] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:51.409791 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 45805.5us
I20260709 12:15:51.370319 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f44bb7910714c46af4e3d0e2076e7c5" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.371155 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b54dd3a3d61243bea2dae318ccea3958" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.369771 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.410599 30194 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:51.426205 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 46630.7us
I20260709 12:15:51.430018 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:15:51.385233 23679 catalog_manager.cc:5697] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:51.446069 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.447404 30194 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:51.447926 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 503101us
I20260709 12:15:51.449318 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:51.462196 30236 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:51.463788 30236 consensus_queue.cc:1059] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:51.439260 30213 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:51.477666 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:51.478036 23858 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.482992 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "526cc198da134bb590358ed73c23d59e" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:51.483757 25171 raft_consensus.cc:3060] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.486724 30181 raft_consensus.cc:3055] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:51.487207 30181 raft_consensus.cc:740] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:51.488318 30181 consensus_queue.cc:260] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.490034 30181 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:51.483901 23898 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:51.493359 30213 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:51.514036 23785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5038ac294ceb4758b5b5de70f2469d5d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.522179 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.506069 23783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:51.526729 23852 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:51.531132 23899 leader_election.cc:304] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:51.548312 25171 raft_consensus.cc:2468] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:15:51.565456 23934 consensus_queue.cc:237] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:15:51.548102 23824 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.569655 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:51.582592 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:51.587379 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:51.608130 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:51.618775 25114 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:51.626245 30214 raft_consensus.cc:2749] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:51.631955 30165 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:51.656666 25168 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:15:51.659682 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 130260us
I20260709 12:15:51.660799 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:51.661827 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9ef29919aa42aaa2b7545c10e39eff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:51.638816 23681 catalog_manager.cc:5697] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:51.673094 25168 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:51.679766 30199 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:15:51.683041 23681 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 10 to 11, term changed from 11 to 12, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to <none>, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 12 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:51.687100 30220 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:51.701401 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:51.712085 30239 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:51.704486 30165 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:51.723661 25170 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:51.730726 23749 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.731755 30198 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:51.741755 23934 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:51.744987 23749 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.746362 23749 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:51.745626 30229 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:15:51.746311 30232 raft_consensus.cc:2749] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:51.751528 30220 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:15:51.755110 30145 raft_consensus.cc:2749] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:51.770905 30073 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:51.774253 25172 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:51.781455 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 105653us
I20260709 12:15:51.797603 23781 raft_consensus.cc:1240] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Rejecting Update request from peer 438159d8de3c4027bdcf57eb81dedf86 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:51.805692 23783 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:51.812965 30194 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:51.826972 23934 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:51.866823 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 51713.2us
I20260709 12:15:51.873399 25175 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:15:51.874306 30273 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.089s	user 0.004s	sys 0.000s
I20260709 12:15:51.878580 23783 raft_consensus.cc:1275] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:15:51.881376 30193 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:51.884507 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 98716.9us
I20260709 12:15:51.887523 30229 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:15:51.957355 23860 raft_consensus.cc:1275] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:51.963188 30193 consensus_queue.cc:1048] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:51.984056 25170 raft_consensus.cc:2955] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:52.022974 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 127159us
W20260709 12:15:52.025998 30275 log.cc:927] Time spent T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: Append to log took a long time: real 0.146s	user 0.002s	sys 0.000s
I20260709 12:15:52.030876 30213 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:52.037118 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 126522us
I20260709 12:15:52.036005 25168 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:52.072129 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 159605us
I20260709 12:15:52.119109 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 053977846bfa4adf9703ea669a81f41d with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:52.126639 23776 raft_consensus.cc:2955] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:52.119256 30214 consensus_queue.cc:1048] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:15:52.136260 23676 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 11 to 12, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:52.158433 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 053977846bfa4adf9703ea669a81f41d with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:57978
W20260709 12:15:52.173938 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 60418.1us
I20260709 12:15:52.176743 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:52.178372 30255 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:52.184780 30291 raft_consensus.cc:493] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:15:52.207289 30291 raft_consensus.cc:515] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:52.209824 30291 leader_election.cc:290] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:52.222028 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e1031da3c683416ebfc283d98b45867d with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:52.228335 23681 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 10 to 11, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 12 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:52.239389 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:52.247308 23777 raft_consensus.cc:2955] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:52.265676 23677 catalog_manager.cc:5697] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 14 to 15, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:52.273806 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef461146a2924dba8b8e2906d76859f1" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:52.305670 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet fc09548ba8eb4f01bf6d74a33d072dda with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:50860
I20260709 12:15:52.345324 23898 leader_election.cc:304] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:52.356305 30291 raft_consensus.cc:2749] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:52.377421 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 241887us
I20260709 12:15:52.378407 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fc09548ba8eb4f01bf6d74a33d072dda with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:52.408342 30294 tablet_replica.cc:333] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:15:52.444434 30295 tablet_replica.cc:333] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:15:52.445933 30295 raft_consensus.cc:2243] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:52.447153 30295 raft_consensus.cc:2272] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:52.451887 30294 raft_consensus.cc:2243] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:52.453143 30294 raft_consensus.cc:2272] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:52.464809 30294 ts_tablet_manager.cc:1916] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:15:52.486838 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 67030.9us
I20260709 12:15:52.511715 30294 ts_tablet_manager.cc:1929] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:15:52.512375 30294 log.cc:1199] T 053977846bfa4adf9703ea669a81f41d P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/053977846bfa4adf9703ea669a81f41d
I20260709 12:15:52.514806 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 053977846bfa4adf9703ea669a81f41d (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
W20260709 12:15:52.528703 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 87498.2us
I20260709 12:15:52.514940 30295 ts_tablet_manager.cc:1916] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:52.548998 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet e1031da3c683416ebfc283d98b45867d with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:35466
I20260709 12:15:52.567528 30299 tablet_replica.cc:333] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:15:52.568938 30299 raft_consensus.cc:2243] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:52.570016 30299 raft_consensus.cc:2272] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:52.581962 30299 ts_tablet_manager.cc:1916] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:52.603096 30299 ts_tablet_manager.cc:1929] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.14
I20260709 12:15:52.603652 30299 log.cc:1199] T e1031da3c683416ebfc283d98b45867d P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/e1031da3c683416ebfc283d98b45867d
I20260709 12:15:52.605559 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet e1031da3c683416ebfc283d98b45867d (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:52.689153 30295 ts_tablet_manager.cc:1929] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 12:15:52.689728 30295 log.cc:1199] T fc09548ba8eb4f01bf6d74a33d072dda P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/fc09548ba8eb4f01bf6d74a33d072dda
I20260709 12:15:52.691805 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet fc09548ba8eb4f01bf6d74a33d072dda (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:52.695675 30298 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.696321 30298 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:52.700245 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:52.700985 23934 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 7.
I20260709 12:15:52.702425 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:52.703428 30194 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:52.703881 30194 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.704221 30194 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:52.704622 30298 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:52.707662 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:52.708388 25168 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 6.
I20260709 12:15:52.732163 30194 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:52.734333 30194 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:52.756223 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:52.761471 23934 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:52.770051 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:52.770737 25168 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:15:52.789423 25168 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:52.801717 23934 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 8.
I20260709 12:15:52.803759 23824 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:52.805189 30194 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:52.805738 30194 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:52.806675 30194 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:52.834287 23683 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:52.837920 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:52.966980 30280 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.967566 30280 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:52.972816 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:15:52.973493 23934 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:52.974866 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25" is_pre_election: true
I20260709 12:15:52.975548 23781 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:15:52.978004 23825 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:52.978943 30194 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:52.979341 30194 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.979707 30194 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:52.998870 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:15:52.994905 30280 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:53.049634 30194 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:53.054950 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:53.055624 23781 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:53.060652 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:53.061338 23934 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:53.066852 23781 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:53.070824 23777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:53.079988 23777 raft_consensus.cc:606] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:53.080147 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:53.080857 25168 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:15:53.085633 23929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:53.086300 23929 raft_consensus.cc:606] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:53.088932 23934 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:53.090161 23824 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:53.090619 23929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:53.091240 23929 raft_consensus.cc:606] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:53.091692 30298 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:53.092247 30298 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:53.092594 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:15:53.094761 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:53.093916 30298 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:53.108749 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:53.147456 30194 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:53.156044 25168 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:15:53.205844 23683 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 12 to 13, leader changed from <none> to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:53.222251 25168 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:15:53.223829 30298 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:53.242108 23931 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:15:53.246778 30298 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:53.271898 30214 raft_consensus.cc:993] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:53.272485 30214 raft_consensus.cc:1081] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:53.275095 23856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:53.275715 23856 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:53.276052 23856 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.294953 30211 raft_consensus.cc:993] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:53.295706 30211 raft_consensus.cc:1081] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:53.309728 23860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:53.310343 23860 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:53.310735 23860 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:53.325268 23856 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.338340 23860 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.361522 23777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:53.362172 23777 raft_consensus.cc:3055] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:53.362546 23777 raft_consensus.cc:740] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:53.363379 23777 consensus_queue.cc:260] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.364557 23777 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:53.368561 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:53.369880 23856 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:53.370715 23781 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.382246 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:53.390684 23931 raft_consensus.cc:3055] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:53.391136 23931 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:53.392050 23931 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.394081 23931 raft_consensus.cc:3060] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.402613 23777 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 22.
I20260709 12:15:53.412127 23781 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:53.412575 23825 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:53.414549 23825 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:53.414551 30280 raft_consensus.cc:2804] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:53.415422 30181 raft_consensus.cc:2804] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:53.415397 30280 raft_consensus.cc:697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 22 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:53.417893 30280 consensus_queue.cc:237] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.421043 23860 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 22 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:53.423566 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b23f5ca4f084b8e96cc4a3099e99074" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:53.424304 23929 raft_consensus.cc:3060] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:53.436247 23929 raft_consensus.cc:2468] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 22.
I20260709 12:15:53.446820 23931 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:53.468358 30181 raft_consensus.cc:697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:53.515895 30181 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:15:53.536262 25168 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:53.537955 30194 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:53.542155 25168 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:53.585320 23683 catalog_manager.cc:5697] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 21 to 22, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 22 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:53.587193 23683 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 16 to 17, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:53.592276 23777 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:53.595379 30194 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:53.600168 30199 raft_consensus.cc:993] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:15:53.600688 30199 raft_consensus.cc:1081] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:15:53.602837 25171 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44508
I20260709 12:15:53.603474 25171 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:53.603876 25171 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:53.638774 25171 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:53.642370 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:53.643071 23929 raft_consensus.cc:3055] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:53.643357 23929 raft_consensus.cc:740] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:53.644134 23929 consensus_queue.cc:260] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:53.646631 25171 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:53.647198 23929 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:53.647930 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:53.650246 23856 raft_consensus.cc:3060] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:53.704250 23929 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:53.705556 23856 raft_consensus.cc:2468] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:15:53.706313 23777 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Committing config change with OpId 11.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:53.713016 25113 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:53.714936 30268 raft_consensus.cc:2804] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:53.715472 30268 raft_consensus.cc:697] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:15:53.717334 30268 consensus_queue.cc:237] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:53.729401 23934 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:53.731299 30181 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:53.757761 23683 catalog_manager.cc:5697] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 12, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:53.960331 23781 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 22 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:15:53.960685 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 84712.3us
I20260709 12:15:54.005057 23776 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:54.006072 30307 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:15:54.049741 23825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 43919.3us
I20260709 12:15:54.067394 30307 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:54.094905 23931 raft_consensus.cc:1275] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:15:54.143446 30317 consensus_queue.cc:1048] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:54.154439 23860 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:54.159801 23934 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:54.161595 30267 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:54.193938 23856 raft_consensus.cc:1275] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:15:54.223817 23929 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:54.229827 25171 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:54.238905 30268 consensus_queue.cc:1048] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:54.239148 23929 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:15:54.272833 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.275337 30280 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:54.306890 30194 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:54.309156 30194 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:54.313000 30326 log.cc:927] Time spent T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:15:54.362376 30280 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Committing config change with OpId 13.13: config changed from index 11 to 13, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:54.390762 25171 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Committing config change with OpId 13.13: config changed from index 11 to 13, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:54.400174 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 98e5bc86804f4f979f58779caa696814 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:54.416975 23931 raft_consensus.cc:2955] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Committing config change with OpId 13.13: config changed from index 11 to 13, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:54.421694 23677 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 11 to 13, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 13 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:54.514670 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 98e5bc86804f4f979f58779caa696814 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:54.543463 30332 tablet_replica.cc:333] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:15:54.561692 23856 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:54.562364 23856 raft_consensus.cc:606] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Received request to transfer leadership to TS 9e0bf56d58c5480a8d088c85566558da
I20260709 12:15:54.564159 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:15:54.566177 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:54.572367 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:54.616786 30332 raft_consensus.cc:2243] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:54.617741 30332 raft_consensus.cc:2272] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:54.621795 30332 ts_tablet_manager.cc:1916] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:54.682603 30332 ts_tablet_manager.cc:1929] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.12
I20260709 12:15:54.715704 30332 log.cc:1199] T 98e5bc86804f4f979f58779caa696814 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/98e5bc86804f4f979f58779caa696814
I20260709 12:15:54.725987 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 98e5bc86804f4f979f58779caa696814 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:15:55.003770 30317 raft_consensus.cc:993] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:55.004343 30317 raft_consensus.cc:1081] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Signalling peer 9e0bf56d58c5480a8d088c85566558da to start an election
I20260709 12:15:55.006140 23931 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
 from {username='slave'} at 127.0.0.1:45620
I20260709 12:15:55.006699 23931 raft_consensus.cc:493] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:55.007037 23931 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:55.016497 23931 raft_consensus.cc:515] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:55.019398 23859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:55.020076 23859 raft_consensus.cc:3055] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:55.020378 23859 raft_consensus.cc:740] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:55.021181 23859 consensus_queue.cc:260] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:55.022364 23859 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:55.060611 23931 leader_election.cc:290] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:55.064018 23859 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:55.065538 23900 leader_election.cc:304] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:15:55.060802 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de3c70defd864769b26b52b7f36548cd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:55.067028 30292 raft_consensus.cc:2804] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:55.067646 30292 raft_consensus.cc:697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:55.066872 25168 raft_consensus.cc:3060] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:55.068619 30292 consensus_queue.cc:237] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:55.079416 25168 raft_consensus.cc:2468] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 9.
I20260709 12:15:55.306754 23683 catalog_manager.cc:5697] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 8 to 9, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 9 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:55.554349 23859 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:55.558367 30292 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:55.647948 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.677412 25171 raft_consensus.cc:1275] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:55.679708 30335 consensus_queue.cc:1048] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:15:55.752388 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 43892.7us
I20260709 12:15:56.143615 23929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:46470
I20260709 12:15:56.144340 23929 raft_consensus.cc:606] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:56.155843 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:15:56.157946 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:56.158818 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:56.219909 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 83003.3us
W20260709 12:15:56.221325 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 67671.1us
W20260709 12:15:56.235880 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 101786us
W20260709 12:15:56.239753 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 99908us
W20260709 12:15:56.249094 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 101431us
I20260709 12:15:56.254793 30292 raft_consensus.cc:993] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:56.255448 30292 raft_consensus.cc:1081] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:56.258363 23858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:15:56.258955 23858 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:56.259298 23858 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:56.271178 23858 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:56.280380 23858 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Requested vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:56.288344 23776 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:56.290104 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:56.291285 25171 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:56.293895 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:56.294564 23929 raft_consensus.cc:3055] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:56.294855 23929 raft_consensus.cc:740] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:15:56.295734 23929 consensus_queue.cc:260] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:56.299678 23929 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:56.318143 25171 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:56.320125 23824 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:56.322888 30346 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:56.323449 30346 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:56.324429 30346 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:56.342357 23929 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 11.
I20260709 12:15:56.348294 23852 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:56.350726 30358 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:56.363794 23683 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 10 to 11, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 11 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.392560 23929 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:15:56.396452 23749 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8341d6ef4764792b1bd6ff196e8af38. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:56.405784 30314 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:56.409348 23852 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.408819 30333 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.432572 23934 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index -1 to 12, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.465745 23852 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:56.467533 23681 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 12, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
W20260709 12:15:56.489915 23824 consensus_peers.cc:597] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78f1c4dc751a4f4a93a446a785edda51. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:56.492770 23934 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:56.514202 30355 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:56.627678 23776 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:56.629426 30355 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:56.728336 30194 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.741178 23777 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.755421 23776 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:56.756810 23681 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 19 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.769039 25171 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:56.774734 30280 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:15:56.789134 23934 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.886035 23929 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:56.888388 30355 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:56.936702 30379 ts_tablet_manager.cc:933] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:15:56.962662 23749 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 055432f1640d4d5a8a4ec2355e0b89ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:56.966467 23934 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:56.969986 23859 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:56.971903 30314 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:56.975800 30381 ts_tablet_manager.cc:933] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:15:56.967257 30379 tablet_copy_client.cc:323] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:15:56.979565 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet d8341d6ef4764792b1bd6ff196e8af38 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:15:56.980218 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet d8341d6ef4764792b1bd6ff196e8af38 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-d8341d6ef4764792b1bd6ff196e8af38
I20260709 12:15:56.989965 30106 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:56.996562 30381 tablet_copy_client.cc:323] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:15:56.999521 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 78f1c4dc751a4f4a93a446a785edda51 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:15:57.000135 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 78f1c4dc751a4f4a93a446a785edda51 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-78f1c4dc751a4f4a93a446a785edda51
I20260709 12:15:57.044781 23859 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:57.077783 23795 tablet_copy_source_session.cc:215] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:57.107076 23870 tablet_copy_source_session.cc:215] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:57.109093 30350 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:57.175320 30381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78f1c4dc751a4f4a93a446a785edda51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.227111 30379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8341d6ef4764792b1bd6ff196e8af38. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:57.253868 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 78595.8us
I20260709 12:15:57.265558 23683 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:57.265758 23934 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:57.287165 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:57.321949 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 60179.7us
I20260709 12:15:57.342415 30379 tablet_copy_client.cc:806] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:57.352350 30379 tablet_copy_client.cc:670] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:57.352424 30381 tablet_copy_client.cc:806] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:57.357479 30381 tablet_copy_client.cc:670] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:57.415172 23859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:15:57.415931 23859 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:15:57.424679 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:15:57.426595 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:57.427749 30381 tablet_copy_client.cc:538] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:57.427453 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:57.418200 30379 tablet_copy_client.cc:538] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:57.448565 23929 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:15:57.544812 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 127816us
W20260709 12:15:57.613131 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 61045.9us
W20260709 12:15:57.633966 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 71364.6us
W20260709 12:15:57.652016 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 70400.4us
I20260709 12:15:57.717288 23781 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:57.757740 30381 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:57.769117 30411 ts_tablet_manager.cc:933] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:15:57.776417 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 58679.8us
I20260709 12:15:57.789561 23852 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:57.796851 30411 tablet_copy_client.cc:323] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:15:57.797487 30379 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:57.808027 30390 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:15:57.852348 30352 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:57.847352 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet 055432f1640d4d5a8a4ec2355e0b89ac from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:15:57.863754 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet 055432f1640d4d5a8a4ec2355e0b89ac from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-055432f1640d4d5a8a4ec2355e0b89ac
W20260709 12:15:57.875883 23898 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77aa2cbe928a45d4b1652ca10a80aa99. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:57.894533 23795 tablet_copy_source_session.cc:215] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:15:57.912659 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 276300us
I20260709 12:15:57.921101 30411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055432f1640d4d5a8a4ec2355e0b89ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.943512 30389 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:57.972987 30356 raft_consensus.cc:993] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:57.982865 23859 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:57.979388 30356 raft_consensus.cc:1081] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:15:58.025556 23681 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:58.039968 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:15:58.040791 23780 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.041251 23780 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.057828 23780 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.060135 23780 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 20 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:58.064144 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:58.064906 23926 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.064994 23858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:15:58.068640 23858 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:58.068941 23858 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:58.069677 23858 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.071406 23858 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.076637 23780 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:15:58.113373 30411 tablet_copy_client.cc:806] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:58.114023 30411 tablet_copy_client.cc:670] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:58.118985 23858 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 20.
I20260709 12:15:58.122354 23751 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:58.123502 30333 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:58.125599 30333 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:58.128240 23926 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 20.
I20260709 12:15:58.126641 30333 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.140705 30411 tablet_copy_client.cc:538] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:58.152870 23683 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 19 to 20, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 20 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.181046 30411 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:58.201292 23781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:58.201948 23781 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Received request to transfer leadership
I20260709 12:15:58.223161 23781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:58.223781 23781 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Received request to transfer leadership
W20260709 12:15:58.231417 23669 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 78f1c4dc751a4f4a93a446a785edda51 already in progress
I20260709 12:15:58.240327 30379 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:58.241508 30379 tablet_bootstrap.cc:492] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:58.242468 30379 ts_tablet_manager.cc:1403] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.445s	user 0.053s	sys 0.005s
I20260709 12:15:58.258704 30379 raft_consensus.cc:359] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.260001 30379 raft_consensus.cc:740] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:58.260809 30379 consensus_queue.cc:260] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.294445 23781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57978
I20260709 12:15:58.295187 23781 raft_consensus.cc:606] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Received request to transfer leadership
I20260709 12:15:58.301168 30379 ts_tablet_manager.cc:1434] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.001s
I20260709 12:15:58.303397 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d8341d6ef4764792b1bd6ff196e8af38 received from {username='slave'} at 127.0.0.1:40134
W20260709 12:15:58.303783 23731 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
I20260709 12:15:58.303923 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-d8341d6ef4764792b1bd6ff196e8af38 on tablet d8341d6ef4764792b1bd6ff196e8af38 with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:15:58.312579 23731 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 78f1c4dc751a4f4a93a446a785edda51 already in progress
W20260709 12:15:58.312945 23731 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 78f1c4dc751a4f4a93a446a785edda51 already in progress
I20260709 12:15:58.339823 25175 raft_consensus.cc:1217] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.11->[16.12-16.12]   Dedup: 16.12->[]
I20260709 12:15:58.338361 30426 ts_tablet_manager.cc:933] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:58.385452 30426 tablet_copy_client.cc:323] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.67:40463
I20260709 12:15:58.387708 23944 tablet_copy_service.cc:140] P 9e0bf56d58c5480a8d088c85566558da: Received BeginTabletCopySession request for tablet 77aa2cbe928a45d4b1652ca10a80aa99 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:55680)
I20260709 12:15:58.388340 23944 tablet_copy_service.cc:161] P 9e0bf56d58c5480a8d088c85566558da: Beginning new tablet copy session on tablet 77aa2cbe928a45d4b1652ca10a80aa99 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:55680: session id = 05679ef5e2b74861aad2a08b6a05e276-77aa2cbe928a45d4b1652ca10a80aa99
I20260709 12:15:58.422437 23944 tablet_copy_source_session.cc:215] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:15:58.432008 30426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77aa2cbe928a45d4b1652ca10a80aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:58.438508 30381 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:58.471889 30381 tablet_bootstrap.cc:492] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:58.477484 30381 ts_tablet_manager.cc:1403] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.720s	user 0.052s	sys 0.016s
I20260709 12:15:58.480753 30381 raft_consensus.cc:359] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.481660 30381 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:58.491315 30381 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.494643 30381 ts_tablet_manager.cc:1434] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:15:58.495061 30411 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:58.495281 30426 tablet_copy_client.cc:806] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:15:58.496027 30426 tablet_copy_client.cc:670] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:15:58.496980 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-78f1c4dc751a4f4a93a446a785edda51 received from {username='slave'} at 127.0.0.1:46448
I20260709 12:15:58.498201 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-78f1c4dc751a4f4a93a446a785edda51 on tablet 78f1c4dc751a4f4a93a446a785edda51 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:58.499006 30411 tablet_bootstrap.cc:492] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:58.509114 30426 tablet_copy_client.cc:538] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:15:58.500576 30411 ts_tablet_manager.cc:1403] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.320s	user 0.042s	sys 0.009s
I20260709 12:15:58.542598 30411 raft_consensus.cc:359] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.571844 30411 raft_consensus.cc:740] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:58.558326 23858 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:58.574527 30408 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:58.569909 30426 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:15:58.552225 23931 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:58.585355 30408 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:58.589578 25175 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 LEARNER]: Advancing to term 20
I20260709 12:15:58.573771 30411 consensus_queue.cc:260] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.614276 25175 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:58.615458 30411 ts_tablet_manager.cc:1434] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.001s
I20260709 12:15:58.616497 30388 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:58.617677 23795 tablet_copy_service.cc:342] P 0ee4ab0747914e7abc642312517b5e25: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-055432f1640d4d5a8a4ec2355e0b89ac received from {username='slave'} at 127.0.0.1:40134
I20260709 12:15:58.618139 23795 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-055432f1640d4d5a8a4ec2355e0b89ac on tablet 055432f1640d4d5a8a4ec2355e0b89ac with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:58.650035 30408 raft_consensus.cc:993] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:58.651643 30408 raft_consensus.cc:1081] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:58.657258 23859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:58.693866 25171 raft_consensus.cc:1217] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 12:15:58.701972 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:58.702656 23859 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:58.703137 23859 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:58.694001 30350 raft_consensus.cc:1064] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:15:58.707930 30350 raft_consensus.cc:1068] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Unable to promote non-voter 05679ef5e2b74861aad2a08b6a05e276: Service unavailable: leader transfer in progress
W20260709 12:15:58.715618 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 43594.6us
I20260709 12:15:58.726599 30408 raft_consensus.cc:1064] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:15:58.727048 30408 raft_consensus.cc:1068] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Unable to promote non-voter 05679ef5e2b74861aad2a08b6a05e276: Service unavailable: leader transfer in progress
I20260709 12:15:58.766314 23859 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.792096 23781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:58.792730 23781 raft_consensus.cc:3055] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:58.793079 23781 raft_consensus.cc:740] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:58.793865 23781 consensus_queue.cc:260] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.796399 23781 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:58.808282 23781 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 21.
I20260709 12:15:58.809456 23825 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:58.810513 30355 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:58.811051 30355 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:58.812983 30355 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.816989 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:15:58.817684 23931 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:58.819698 23859 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 21 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:15:58.826735 23931 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 21.
I20260709 12:15:58.834975 30350 raft_consensus.cc:1064] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:58.839700 30350 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:58.859812 25175 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:58.860404 23859 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:58.862138 30333 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:58.867530 23934 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:58.874244 30401 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:58.875173 30408 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:58.878602 30426 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:15:58.879647 30426 tablet_bootstrap.cc:492] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:15:58.880414 30426 ts_tablet_manager.cc:1403] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.311s	user 0.046s	sys 0.001s
I20260709 12:15:58.898512 23679 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 20 to 21, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 21 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.931082 30401 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:58.934368 23852 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:58.883523 30426 raft_consensus.cc:359] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.951393 30426 raft_consensus.cc:740] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:15:58.952226 30426 consensus_queue.cc:260] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:15:58.953547 23931 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:58.956663 23781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
mode: GRACEFUL
new_leader_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:37604
I20260709 12:15:58.960865 23781 raft_consensus.cc:606] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Received request to transfer leadership to TS 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:15:58.962216 30426 ts_tablet_manager.cc:1434] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 12:15:58.962415 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:15:58.964149 23944 tablet_copy_service.cc:342] P 9e0bf56d58c5480a8d088c85566558da: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-77aa2cbe928a45d4b1652ca10a80aa99 received from {username='slave'} at 127.0.0.1:55680
I20260709 12:15:58.964586 23944 tablet_copy_service.cc:434] P 9e0bf56d58c5480a8d088c85566558da: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-77aa2cbe928a45d4b1652ca10a80aa99 on tablet 77aa2cbe928a45d4b1652ca10a80aa99 with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:15:58.964685 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:58.965556 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:58.989413 25175 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.050936 23681 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 12 to 13, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.126684 30350 raft_consensus.cc:993] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:59.127185 30350 raft_consensus.cc:1081] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:15:59.148453 23852 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56648
I20260709 12:15:59.149088 23852 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:59.149482 23852 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:59.168501 23852 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.170303 23852 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:15:59.176818 25171 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:15:59.177414 25171 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:59.194319 25171 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:59.195803 23824 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:15:59.196856 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d79b6f8bf5274da7a3f667b9988c5a84" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:15:59.197546 23776 raft_consensus.cc:3055] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:59.197862 23776 raft_consensus.cc:740] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:15:59.198614 23776 consensus_queue.cc:260] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.202700 23776 raft_consensus.cc:3060] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:59.197662 30355 raft_consensus.cc:2804] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:59.205286 30355 raft_consensus.cc:697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:15:59.206275 30355 consensus_queue.cc:237] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.222760 23776 raft_consensus.cc:2468] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:15:59.234282 23781 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:59.247243 30356 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:15:59.249382 23776 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.261354 25171 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:59.264678 30401 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:59.269109 25175 raft_consensus.cc:1217] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 12:15:59.290019 23931 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:59.292145 25171 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 LEARNER]: Advancing to term 21
I20260709 12:15:59.291543 30355 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:15:59.312539 23677 catalog_manager.cc:5697] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.327885 30408 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:15:59.330341 30445 log.cc:927] Time spent T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:15:59.340564 25171 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:15:59.347334 30355 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:59.364109 25171 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.435395 30355 raft_consensus.cc:1064] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:59.437976 30355 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.451920 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8341d6ef4764792b1bd6ff196e8af38 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:15:59.454631 30333 raft_consensus.cc:1064] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:59.457768 30333 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.460386 23679 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 13 to 14, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.463045 23934 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:59.479089 30350 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:15:59.513996 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet d8341d6ef4764792b1bd6ff196e8af38 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:35466
I20260709 12:15:59.533093 23931 raft_consensus.cc:2955] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.14: config changed from index 13 to 14, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.555449 30452 tablet_replica.cc:333] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:15:59.556623 30452 raft_consensus.cc:2243] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 12:15:59.557598 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 82209.7us
I20260709 12:15:59.585640 23781 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:59.587714 30356 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:15:59.587270 30452 raft_consensus.cc:2272] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:59.615924 25175 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:59.620389 30452 ts_tablet_manager.cc:1916] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:15:59.621171 23931 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:59.654260 30355 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:59.656315 30396 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:59.659336 25175 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:59.667188 30350 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:59.673895 23934 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:59.675710 30401 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:59.700829 23852 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:59.699826 30355 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 13 to 16, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.725050 25175 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:15:59.724623 23781 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 13 to 16, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.728725 23679 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 13 to 16, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.738207 30410 raft_consensus.cc:1064] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:15:59.737622 30350 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.741317 30410 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.749763 23934 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.809892 30333 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:59.813282 23780 raft_consensus.cc:1275] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:15:59.814796 23931 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 13 to 16, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.817485 30454 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:59.832118 23858 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.841832 25171 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.853981 23858 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:59.856637 30389 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:59.857702 23781 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:59.864063 30386 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:59.865711 25175 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:59.867324 30386 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:59.863379 23676 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.881767 25168 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 13 to 16, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } attrs { replace: true } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.911298 30410 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.916124 23777 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.921490 25172 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.947365 23676 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:59.950459 23852 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } attrs { replace: true } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:15:59.959527 30454 consensus_queue.cc:1048] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:59.968714 23926 consensus_queue.cc:237] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:15:59.997963 23934 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:00.007054 23776 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:00.007776 25174 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:00.018983 30451 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:00.022825 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 64381us
I20260709 12:16:00.027324 30396 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:00.041615 30386 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.054164 30452 ts_tablet_manager.cc:1929] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.13
I20260709 12:16:00.055397 30452 log.cc:1199] T d8341d6ef4764792b1bd6ff196e8af38 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/d8341d6ef4764792b1bd6ff196e8af38
I20260709 12:16:00.056396 23777 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.081595 23668 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 77aa2cbe928a45d4b1652ca10a80aa99 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:00.089216 23860 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.099018 30356 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:00.099620 23679 catalog_manager.cc:5697] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.111608 25170 raft_consensus.cc:1275] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:00.113399 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet d8341d6ef4764792b1bd6ff196e8af38 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:16:00.121683 23849 tablet_service.cc:1558] Processing DeleteTablet for tablet 77aa2cbe928a45d4b1652ca10a80aa99 with delete_type TABLET_DATA_TOMBSTONED (TS 438159d8de3c4027bdcf57eb81dedf86 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:35466
I20260709 12:16:00.122933 30451 consensus_queue.cc:1048] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:00.125469 30452 tablet_replica.cc:333] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:00.126613 30452 raft_consensus.cc:2243] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.127441 30452 raft_consensus.cc:2272] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.131798 25170 raft_consensus.cc:2955] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.150842 30452 ts_tablet_manager.cc:1916] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:00.209220 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 109280us
I20260709 12:16:00.223641 30452 ts_tablet_manager.cc:1929] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.11
I20260709 12:16:00.224153 30452 log.cc:1199] T 77aa2cbe928a45d4b1652ca10a80aa99 P 438159d8de3c4027bdcf57eb81dedf86: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-1-root/wals/77aa2cbe928a45d4b1652ca10a80aa99
I20260709 12:16:00.225657 23670 catalog_manager.cc:5028] TS 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): tablet 77aa2cbe928a45d4b1652ca10a80aa99 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:16:00.261426 30465 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 LEADER]: Committing config change with OpId 21.17: config changed from index 16 to 17, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.289273 23670 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 78f1c4dc751a4f4a93a446a785edda51 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:00.301931 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:00.300287 25170 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.297775 23676 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 16 to 17, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New cstate: current_term: 21 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:00.344089 23766 tablet_service.cc:1558] Processing DeleteTablet for tablet 78f1c4dc751a4f4a93a446a785edda51 with delete_type TABLET_DATA_TOMBSTONED (TS 0ee4ab0747914e7abc642312517b5e25 not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:57978
I20260709 12:16:00.353888 23934 raft_consensus.cc:2955] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Committing config change with OpId 21.17: config changed from index 16 to 17, VOTER 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
W20260709 12:16:00.415824 23749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 57109.5us
I20260709 12:16:00.483428 23854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:16:00.484153 23854 raft_consensus.cc:606] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Received request to transfer leadership to TS 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:16:00.487286 23854 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
mode: GRACEFUL
new_leader_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:47418
I20260709 12:16:00.487892 23854 raft_consensus.cc:606] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Received request to transfer leadership to TS 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:00.489061 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:00.490916 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:00.491773 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:00.493747 30473 tablet_replica.cc:333] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:00.545517 23780 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.565272 25172 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:16:00.569640 23856 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:16:00.576469 30470 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:16:00.578568 30401 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:00.583985 30473 raft_consensus.cc:2243] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.584826 30473 raft_consensus.cc:2272] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.589521 30476 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.598294 30473 ts_tablet_manager.cc:1916] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:00.606114 25168 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.610498 23856 raft_consensus.cc:2955] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:00.615722 30472 raft_consensus.cc:993] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:16:00.616302 30472 raft_consensus.cc:1081] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Signalling peer 05679ef5e2b74861aad2a08b6a05e276 to start an election
I20260709 12:16:00.621385 25172 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214"
dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
 from {username='slave'} at 127.0.0.1:44500
I20260709 12:16:00.621990 25172 raft_consensus.cc:493] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:00.622344 25172 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.630235 25172 raft_consensus.cc:515] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.635700 30456 raft_consensus.cc:993] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: : Instructing follower 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:16:00.636283 30456 raft_consensus.cc:1081] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Signalling peer 0ee4ab0747914e7abc642312517b5e25 to start an election
I20260709 12:16:00.637461 23776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
I20260709 12:16:00.638137 23776 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.649325 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6d3d4f634284532bbf6ad7d4ac19214" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:00.650032 23854 raft_consensus.cc:3055] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:00.650323 23854 raft_consensus.cc:740] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:16:00.651094 23854 consensus_queue.cc:260] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.652612 23854 raft_consensus.cc:3060] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.657191 25172 leader_election.cc:290] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:00.659567 23780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda"
dest_uuid: "0ee4ab0747914e7abc642312517b5e25"
 from {username='slave'} at 127.0.0.1:43422
I20260709 12:16:00.660179 23780 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:00.660595 23780 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:00.660082 23669 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 055432f1640d4d5a8a4ec2355e0b89ac with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:00.665333 23776 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:16:00.667019 25114 leader_election.cc:304] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25; no voters: 
I20260709 12:16:00.668041 30404 raft_consensus.cc:2804] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:00.668555 30404 raft_consensus.cc:697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:00.669384 30404 consensus_queue.cc:237] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.674278 23780 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.677852 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:00.678493 23852 raft_consensus.cc:3055] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:16:00.679257 23852 raft_consensus.cc:740] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:16:00.681649 23852 consensus_queue.cc:260] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.682934 23852 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:00.684865 23677 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index 12 to 13, VOTER 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) evicted. New cstate: current_term: 16 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:00.695322 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "0ee4ab0747914e7abc642312517b5e25" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:16:00.678256 23780 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:00.696126 25168 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:00.699995 23915 tablet_service.cc:1558] Processing DeleteTablet for tablet 055432f1640d4d5a8a4ec2355e0b89ac with delete_type TABLET_DATA_TOMBSTONED (TS 9e0bf56d58c5480a8d088c85566558da not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:50860
I20260709 12:16:00.717687 23854 raft_consensus.cc:2468] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:16:00.723441 23852 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:16:00.738039 23751 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:00.739538 30401 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:16:00.740339 30401 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Becoming Leader. State: Replica: 0ee4ab0747914e7abc642312517b5e25, State: Running, Role: LEADER
I20260709 12:16:00.746313 30401 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:00.753692 23677 catalog_manager.cc:5697] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 17 to 18, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 18 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:00.775950 30478 tablet_replica.cc:333] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:00.777247 30478 raft_consensus.cc:2243] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:00.789629 30478 raft_consensus.cc:2272] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:00.778759 25168 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ee4ab0747914e7abc642312517b5e25 in term 13.
I20260709 12:16:00.795758 30478 ts_tablet_manager.cc:1916] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:00.806985 23677 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: term changed from 12 to 13, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65). New cstate: current_term: 13 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.809257 30473 ts_tablet_manager.cc:1929] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.16
I20260709 12:16:00.833810 30473 log.cc:1199] T 78f1c4dc751a4f4a93a446a785edda51 P 0ee4ab0747914e7abc642312517b5e25: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-0-root/wals/78f1c4dc751a4f4a93a446a785edda51
I20260709 12:16:00.841289 23669 catalog_manager.cc:5028] TS 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): tablet 78f1c4dc751a4f4a93a446a785edda51 (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:16:00.874397 30478 ts_tablet_manager.cc:1929] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.12
I20260709 12:16:00.897063 30478 log.cc:1199] T 055432f1640d4d5a8a4ec2355e0b89ac P 9e0bf56d58c5480a8d088c85566558da: Deleting WAL directory at /tmp/dist-test-taskDH3MV6/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599102143852-22629-0/minicluster-data/ts-2-root/wals/055432f1640d4d5a8a4ec2355e0b89ac
I20260709 12:16:00.898985 23668 catalog_manager.cc:5028] TS 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): tablet 055432f1640d4d5a8a4ec2355e0b89ac (table test-workload [id=ca3d2698d92e4d56a779e75f45739ac7]) successfully deleted
I20260709 12:16:01.103264 23854 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:16:01.106655 30404 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:01.180688 23776 raft_consensus.cc:1275] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:16:01.193449 30475 consensus_queue.cc:1048] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:01.209399 23854 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:16:01.214985 30474 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:01.259212 25172 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:16:01.263725 30474 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:01.620455 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 75785.2us
W20260709 12:16:01.676036 23751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 125580us
I20260709 12:16:01.773846 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:01.925194 23781 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:01.951484 23934 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:16:01.953907 23749 consensus_peers.cc:597] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c395cd7fe30b41869bce6dac359f2cf8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:01.955260 23854 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 0ee4ab0747914e7abc642312517b5e25: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:16:01.956389 30495 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:16:01.958343 30491 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:16:01.969300 22629 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:01.971298 22629 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:01.972429 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:02.033466 23934 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:16:02.008852 30491 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
W20260709 12:16:02.167199 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 76899us
I20260709 12:16:02.201318 23676 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 reported cstate change: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 14 leader_uuid: "0ee4ab0747914e7abc642312517b5e25" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.254518 23860 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:16:02.364434 22629 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:02.381028 30510 ts_tablet_manager.cc:933] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:16:02.410943 30510 tablet_copy_client.cc:323] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.65:35817
I20260709 12:16:02.413305 23795 tablet_copy_service.cc:140] P 0ee4ab0747914e7abc642312517b5e25: Received BeginTabletCopySession request for tablet c395cd7fe30b41869bce6dac359f2cf8 from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:40134)
I20260709 12:16:02.413983 23795 tablet_copy_service.cc:161] P 0ee4ab0747914e7abc642312517b5e25: Beginning new tablet copy session on tablet c395cd7fe30b41869bce6dac359f2cf8 from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:40134: session id = 05679ef5e2b74861aad2a08b6a05e276-c395cd7fe30b41869bce6dac359f2cf8
I20260709 12:16:02.426203 23795 tablet_copy_source_session.cc:215] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:02.430163 30510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c395cd7fe30b41869bce6dac359f2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:02.470074 30510 tablet_copy_client.cc:806] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:02.470867 30510 tablet_copy_client.cc:670] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:02.484499 30510 tablet_copy_client.cc:538] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:02.514070 30510 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
I20260709 12:16:02.519129 22629 tablet_server.cc:179] TabletServer@127.22.25.65:0 shutting down...
I20260709 12:16:02.571839 22629 tablet_copy_service.cc:362] P 0ee4ab0747914e7abc642312517b5e25: Destroying tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c395cd7fe30b41869bce6dac359f2cf8 due to service shutdown
I20260709 12:16:02.572412 22629 tablet_copy_service.cc:434] P 0ee4ab0747914e7abc642312517b5e25: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-c395cd7fe30b41869bce6dac359f2cf8 on tablet c395cd7fe30b41869bce6dac359f2cf8 with peer 05679ef5e2b74861aad2a08b6a05e276
W20260709 12:16:02.630616 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.674451 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.701987 23899 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.705251 23825 consensus_peers.cc:597] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.707237 23825 consensus_peers.cc:597] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.707865 23899 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.721221 23899 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.722071 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.722733 23899 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.726742 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.727563 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.738476 30510 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:16:02.741931 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.742707 23825 consensus_peers.cc:597] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.744771 30510 tablet_bootstrap.cc:492] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:16:02.745674 30510 ts_tablet_manager.cc:1403] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.232s	user 0.041s	sys 0.018s
W20260709 12:16:02.748291 23825 consensus_peers.cc:597] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.757318 30510 raft_consensus.cc:359] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:02.775568 30510 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:16:02.778434 30510 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:02.777697 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:02.787204 30510 ts_tablet_manager.cc:1434] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.041s	user 0.002s	sys 0.006s
I20260709 12:16:02.788789 22629 tablet_replica.cc:333] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.792200 22629 raft_consensus.cc:2243] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.793056 22629 raft_consensus.cc:2272] T d79b6f8bf5274da7a3f667b9988c5a84 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.796350 22629 tablet_replica.cc:333] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.797305 22629 raft_consensus.cc:2243] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.798026 22629 raft_consensus.cc:2272] T b7caafee6779404494f8df62d888c5ef P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.805794 22629 tablet_replica.cc:333] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.806906 22629 raft_consensus.cc:2243] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.808246 22629 raft_consensus.cc:2272] T 349b687db5c44196bafc207c26ca3679 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:02.808207 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.818852 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.820060 22629 tablet_replica.cc:333] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.821485 22629 raft_consensus.cc:2243] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.837941 22629 raft_consensus.cc:2272] T e76b9730b35144689e48d4d145fabb18 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.841812 22629 tablet_replica.cc:333] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
W20260709 12:16:02.845655 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.856012 22629 raft_consensus.cc:2243] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.860361 22629 raft_consensus.cc:2272] T 956a6729b1f34c499670aa77b5896a53 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:02.867710 23825 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.868449 23825 consensus_peers.cc:597] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.869045 23825 consensus_peers.cc:597] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.869439 22629 tablet_replica.cc:333] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
W20260709 12:16:02.869635 23825 consensus_peers.cc:597] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.878013 23899 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.880399 22629 raft_consensus.cc:2243] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:16:02.881058 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.882097 23899 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.897477 22629 raft_consensus.cc:2272] T e2366a1e0cb342ee891c776affbfbca3 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.900589 22629 tablet_replica.cc:333] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.901512 22629 raft_consensus.cc:2243] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.906397 22629 raft_consensus.cc:2272] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:02.904361 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:02.908838 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.910079 22629 tablet_replica.cc:333] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.918937 22629 raft_consensus.cc:2243] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.920576 22629 raft_consensus.cc:2272] T 4e60a1c6520545729d27474d8f237ce5 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.923382 22629 tablet_replica.cc:333] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.924342 22629 raft_consensus.cc:2243] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.925041 22629 raft_consensus.cc:2272] T d6d3d4f634284532bbf6ad7d4ac19214 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:02.927908 23899 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.936878 22629 tablet_replica.cc:333] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.937968 22629 raft_consensus.cc:2243] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.938736 22629 raft_consensus.cc:2272] T 34d6300a13a3401283a46bf3d2a07e82 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:02.940912 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.942042 22629 tablet_replica.cc:333] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.942960 22629 raft_consensus.cc:2243] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.944212 22629 raft_consensus.cc:2272] T d8341d6ef4764792b1bd6ff196e8af38 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.950027 22629 tablet_replica.cc:333] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.955572 22629 raft_consensus.cc:2243] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:02.956946 22629 raft_consensus.cc:2272] T c4369a8cdaaa45beac1359775fa54781 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.959833 22629 tablet_replica.cc:333] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.964355 22629 raft_consensus.cc:2243] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.965339 22629 raft_consensus.cc:2272] T f5bb29269e8a4aa1b79f91e4ed687053 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.969559 25168 raft_consensus.cc:1217] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 12:16:02.980139 22629 tablet_replica.cc:333] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.981185 22629 raft_consensus.cc:2243] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:02.981901 22629 raft_consensus.cc:2272] T 24dc0370e122479081184511cb1224f1 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:02.985090 22629 tablet_replica.cc:333] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:02.986681 22629 raft_consensus.cc:2243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:02.991017 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:02.988071 22629 raft_consensus.cc:2272] T 8f44bb7910714c46af4e3d0e2076e7c5 P 0ee4ab0747914e7abc642312517b5e25 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.001451 22629 tablet_replica.cc:333] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.005213 22629 raft_consensus.cc:2243] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.008016 22629 raft_consensus.cc:2272] T 1b93ab9c24064567b717af205bc2ed2a P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.026592 22629 tablet_replica.cc:333] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.036377 22629 raft_consensus.cc:2243] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.037331 22629 raft_consensus.cc:2272] T 3eeb67716b0e4f49aa1a4bd5815f684b P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.041463 22629 tablet_replica.cc:333] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.043503 22629 raft_consensus.cc:2243] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.044895 22629 raft_consensus.cc:2272] T 20edc48b2f884789ac3daead4b5c3102 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.049222 22629 tablet_replica.cc:333] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.060534 22629 raft_consensus.cc:2243] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.062078 22629 raft_consensus.cc:2272] T fc09548ba8eb4f01bf6d74a33d072dda P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.065224 22629 tablet_replica.cc:333] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.066227 22629 raft_consensus.cc:2243] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.066991 22629 raft_consensus.cc:2272] T e1031da3c683416ebfc283d98b45867d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:03.068094 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:03.078737 22629 tablet_replica.cc:333] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.083196 22629 raft_consensus.cc:2243] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.084156 22629 raft_consensus.cc:2272] T 8f333e0a484248ce88f0905e18500838 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.100258 22629 tablet_replica.cc:333] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.101218 22629 raft_consensus.cc:2243] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.103118 22629 raft_consensus.cc:2272] T c2844521776744308787188de9041ebd P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.105943 22629 tablet_replica.cc:333] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.106846 22629 raft_consensus.cc:2243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.107714 22629 raft_consensus.cc:2272] T 20fc63afa03b45c0bf3f58aae82f3e3d P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.112228 22629 tablet_replica.cc:333] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.124374 22629 raft_consensus.cc:2243] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.137419 22629 raft_consensus.cc:2272] T c395cd7fe30b41869bce6dac359f2cf8 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.144901 22629 tablet_replica.cc:333] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.146318 22629 raft_consensus.cc:2243] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.160116 22629 raft_consensus.cc:2272] T 7f5f207bbfed4f119db8464be0a5a8cf P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.178362 22629 tablet_replica.cc:333] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.184330 22629 raft_consensus.cc:2243] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.192050 22629 raft_consensus.cc:2272] T 34e5e0c10245477d9010ece72d08ae51 P 0ee4ab0747914e7abc642312517b5e25 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.195235 22629 tablet_replica.cc:333] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.204373 22629 raft_consensus.cc:2243] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.205368 22629 raft_consensus.cc:2272] T 861b3b9e09c14ef99794d4c8d50d59ad P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.226308 22629 tablet_replica.cc:333] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.227174 22629 raft_consensus.cc:2243] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.228379 22629 raft_consensus.cc:2272] T 102493ff11494b009f79963bb57be511 P 0ee4ab0747914e7abc642312517b5e25 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.231081 22629 tablet_replica.cc:333] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.231878 22629 raft_consensus.cc:2243] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:03.238576 25114 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 44 similar messages]
I20260709 12:16:03.249153 22629 raft_consensus.cc:2272] T ef461146a2924dba8b8e2906d76859f1 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.252159 22629 tablet_replica.cc:333] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.253012 22629 raft_consensus.cc:2243] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.254175 22629 raft_consensus.cc:2272] T 5ed3fc49956b40aa86095410b7c7e5fa P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.289050 22629 tablet_replica.cc:333] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.290051 22629 raft_consensus.cc:2243] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.291579 22629 raft_consensus.cc:2272] T 0b23f5ca4f084b8e96cc4a3099e99074 P 0ee4ab0747914e7abc642312517b5e25 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.294919 22629 tablet_replica.cc:333] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.295794 22629 raft_consensus.cc:2243] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.299275 22629 raft_consensus.cc:2272] T 643746a7a11243df94746cf1081fc078 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.309676 22629 tablet_replica.cc:333] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.310617 22629 raft_consensus.cc:2243] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.311298 22629 raft_consensus.cc:2272] T 5038ac294ceb4758b5b5de70f2469d5d P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.314393 22629 tablet_replica.cc:333] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.315393 22629 raft_consensus.cc:2243] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.316092 22629 raft_consensus.cc:2272] T 71cb2ce294574daa8d7244ae252e8aa6 P 0ee4ab0747914e7abc642312517b5e25 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.341470 22629 tablet_replica.cc:333] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.342713 22629 raft_consensus.cc:2243] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.345511 22629 raft_consensus.cc:2272] T 055432f1640d4d5a8a4ec2355e0b89ac P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.352936 22629 tablet_replica.cc:333] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.366564 22629 raft_consensus.cc:2243] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.386705 22629 raft_consensus.cc:2272] T 3aacc8d80dd2433b94e6d07927304b47 P 0ee4ab0747914e7abc642312517b5e25 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.389761 22629 tablet_replica.cc:333] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.404451 22629 raft_consensus.cc:2243] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.405362 22629 raft_consensus.cc:2272] T 9d72845b5ad4457bb71a7ad5f5152b4c P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.436434 22629 tablet_replica.cc:333] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.439155 22629 raft_consensus.cc:2243] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.439946 22629 raft_consensus.cc:2272] T cc9ef29919aa42aaa2b7545c10e39eff P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.473366 22629 tablet_replica.cc:333] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.474295 22629 raft_consensus.cc:2243] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:16:03.480674 22629 raft_consensus.cc:2272] T ed3c66ee31004577b517e2c4322696b6 P 0ee4ab0747914e7abc642312517b5e25 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.483484 22629 tablet_replica.cc:333] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.487171 22629 raft_consensus.cc:2243] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.487917 22629 raft_consensus.cc:2272] T 0c9d6097cb8d40efa55ff97de5b26130 P 0ee4ab0747914e7abc642312517b5e25 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.490777 22629 tablet_replica.cc:333] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.491529 22629 raft_consensus.cc:2243] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.492553 22629 raft_consensus.cc:2272] T 872245a7005f472bb7f6cfafda680b74 P 0ee4ab0747914e7abc642312517b5e25 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.519613 22629 tablet_replica.cc:333] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.520870 22629 raft_consensus.cc:2243] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.523063 22629 raft_consensus.cc:2272] T b54dd3a3d61243bea2dae318ccea3958 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.526259 22629 tablet_replica.cc:333] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.531132 22629 raft_consensus.cc:2243] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.535282 22629 raft_consensus.cc:2272] T a979dd022fc24afd8765a3cdb5f7fab3 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.543726 22629 tablet_replica.cc:333] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.544576 22629 raft_consensus.cc:2243] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.545876 22629 raft_consensus.cc:2272] T 77aa2cbe928a45d4b1652ca10a80aa99 P 0ee4ab0747914e7abc642312517b5e25 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.548869 22629 tablet_replica.cc:333] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25: stopping tablet replica
I20260709 12:16:03.549850 22629 raft_consensus.cc:2243] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:03.550577 22629 raft_consensus.cc:2272] T bec362a6d01840d39be46f0c3d28c218 P 0ee4ab0747914e7abc642312517b5e25 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:03.781435 30518 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:03.782029 30518 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:16:03.787528 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:03.789103 30518 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:03.789705 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:03.791654 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:03.793045 30451 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:03.900456 30497 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:03.901120 30497 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:03.904783 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:03.908933 30497 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:03.921350 23825 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:03.921984 23825 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:03.924969 30497 raft_consensus.cc:2749] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:04.015329 30451 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.015969 30451 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.018600 30451 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.023331 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:04.032333 23899 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.032825 23899 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.033991 30518 raft_consensus.cc:2749] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:04.076792 30518 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.077421 30518 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.079689 30518 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.081853 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:04.086167 23899 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.086611 23899 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.088690 30451 raft_consensus.cc:2749] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:04.127556 30501 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.128312 30501 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.134163 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.134886 23925 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
W20260709 12:16:04.137158 25114 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.141016 25113 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.142127 30404 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:04.142527 30404 raft_consensus.cc:493] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.142846 30404 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
==================
WARNING: ThreadSanitizer: data race (pid=22629)
  Write of size 4 at 0x7b40000306d8 by main thread:
    #0 kudu::DiagnosticSocket::Close() /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:96:7 (libkudu_util.so+0x49fe03)
    #1 kudu::rpc::AcceptorPool::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:212:3 (libkrpc.so+0x12ad33)
    #2 kudu::rpc::Messenger::ShutdownInternal(kudu::rpc::Messenger::ShutdownMode) /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:239:8 (libkrpc.so+0x16d257)
    #3 kudu::rpc::Messenger::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:206:3 (libkrpc.so+0x16d80b)
    #4 kudu::kserver::KuduServer::Shutdown() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:192:17 (libkserver.so+0x13865)
    #5 kudu::tserver::TabletServer::ShutdownImpl() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:195:17 (libtserver.so+0x25c2ca)
    #6 kudu::tserver::TabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x261a79)
    #7 kudu::tserver::MiniTabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x31c692)
    #8 kudu::cluster::InternalMiniCluster::ShutdownNodes(kudu::cluster::ClusterNodes) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:242:22 (libmini_cluster.so+0xd7a87)
    #9 kudu::cluster::MiniCluster::Shutdown() /apache/dev/git/kudu/src/kudu/mini-cluster/mini_cluster.h:84:5 (auto_leader_rebalancer-test+0x3758e4)
    #10 kudu::master::LeaderRebalancerTest::TearDown() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:116:17 (auto_leader_rebalancer-test+0x363142)
    #11 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #12 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #13 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2682:3 (libgtest.so.1.12.1+0x42a82)
    #14 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #15 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #16 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #17 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #18 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #19 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #20 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #21 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

  Previous read of size 4 at 0x7b40000306d8 by thread T264 (mutexes: write M915773476609850416):
    #0 kudu::DiagnosticSocket::ReceiveResponse(std::__1::vector<kudu::DiagnosticSocket::TcpSocketInfo, std::__1::allocator<kudu::DiagnosticSocket::TcpSocketInfo> >*) const /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:264:3 (libkudu_util.so+0x4a08e1)
    #1 kudu::DiagnosticSocket::Query(kudu::Socket const&, kudu::DiagnosticSocket::TcpSocketInfo*) const /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:134:3 (libkudu_util.so+0x4a1274)
    #2 kudu::rpc::AcceptorPool::GetPendingConnectionsNum(unsigned int*) const /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:241:3 (libkrpc.so+0x12b653)
    #3 kudu::rpc::Messenger::GetPendingConnectionsNum() /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:193:21 (libkrpc.so+0x16d4b1)
    #4 kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1::operator()() const /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:299:51 (libkrpc.so+0x171bb1)
    #5 decltype(std::__1::forward<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(fp)()) std::__1::__invoke<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkrpc.so+0x171b69)
    #6 int std::__1::__invoke_void_return_wrapper<int>::__call<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&>(kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libkrpc.so+0x171af9)
    #7 std::__1::__function::__alloc_func<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1, std::__1::allocator<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1>, int ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkrpc.so+0x171ab1)
    #8 std::__1::__function::__func<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1, std::__1::allocator<kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*)::$_1>, int ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkrpc.so+0x170dad)
I20260709 12:16:04.147684 30501 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
    #9 std::__1::__function::__value_func<int ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x4d3944)
    #10 std::__1::function<int ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x4d38e9)
    #11 kudu::FunctionGauge<int>::value() const /apache/dev/git/kudu/src/kudu/util/metrics.h:1319:12 (libmaster.so+0x4d381a)
    #12 kudu::FunctionGauge<int>::WriteValue(kudu::JsonWriter*) const /apache/dev/git/kudu/src/kudu/util/metrics.h:1323:19 (libmaster.so+0x4d3500)
    #13 kudu::Gauge::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:856:3 (libkudu_util.so+0x380112)
    #14 void kudu::WriteMetricsToJson<std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > >(kudu::JsonWriter*, std::__1::unordered_map<kudu::MetricPrototype const*, scoped_refptr<kudu::Metric>, std::__1::hash<kudu::MetricPrototype const*>, std::__1::equal_to<kudu::MetricPrototype const*>, std::__1::allocator<std::__1::pair<kudu::MetricPrototype const* const, scoped_refptr<kudu::Metric> > > > const&, kudu::MetricJsonOptions const&) /apache/dev/git/kudu/src/kudu/util/metrics.cc:79:7 (libkudu_util.so+0x3851d0)
    #15 kudu::MetricEntity::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:414:3 (libkudu_util.so+0x37d847)
    #16 kudu::MetricRegistry::WriteAsJson(kudu::JsonWriter*, kudu::MetricJsonOptions const&) const /apache/dev/git/kudu/src/kudu/util/metrics.cc:586:7 (libkudu_util.so+0x37eacc)
    #17 kudu::server::DiagnosticsLog::LogMetrics() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:348:3 (libserver_process.so+0x11db17)
    #18 kudu::server::DiagnosticsLog::RunThread() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:228:7 (libserver_process.so+0x11d7f6)
    #19 kudu::server::DiagnosticsLog::Start()::$_0::operator()() const /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:145:46 (libserver_process.so+0x11f381)
    #20 decltype(std::__1::forward<kudu::server::DiagnosticsLog::Start()::$_0&>(fp)()) std::__1::__invoke<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libserver_process.so+0x11f339)
    #21 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::server::DiagnosticsLog::Start()::$_0&>(kudu::server::DiagnosticsLog::Start()::$_0&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libserver_process.so+0x11f2c9)
    #22 std::__1::__function::__alloc_func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libserver_process.so+0x11f291)
    #23 std::__1::__function::__func<kudu::server::DiagnosticsLog::Start()::$_0, std::__1::allocator<kudu::server::DiagnosticsLog::Start()::$_0>, void ()>::operator()() /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libserver_process.so+0x11e58d)
    #24 std::__1::__function::__value_func<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (auto_leader_rebalancer-test+0x388634)
    #25 std::__1::function<void ()>::operator()() const /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (auto_leader_rebalancer-test+0x388479)
    #26 kudu::Thread::SuperviseThread(void*) /apache/dev/git/kudu/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x40e83a)

  Location is heap block of size 248 at 0x7b4000030600 allocated by main thread:
    #0 operator new(unsigned long) /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64 (auto_leader_rebalancer-test+0x3535e7)
    #1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/new:253:10 (libkrpc.so+0x130946)
    #2 std::__1::allocator<std::__1::__shared_ptr_emplace<kudu::rpc::AcceptorPool, std::__1::allocator<kudu::rpc::AcceptorPool> > >::allocate(unsigned long) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/memory:1789:34 (libkrpc.so+0x17a5e1)
    #3 std::__1::enable_if<!(is_array<kudu::rpc::AcceptorPool>::value), std::__1::shared_ptr<kudu::rpc::AcceptorPool> >::type std::__1::make_shared<kudu::rpc::AcceptorPool, kudu::rpc::Messenger*, kudu::Socket*, kudu::Sockaddr&, int const&>(kudu::rpc::Messenger*&&, kudu::Socket*&&, kudu::Sockaddr&, int const&) /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/memory:4290:45 (libkrpc.so+0x172a25)
    #4 kudu::rpc::Messenger::AddAcceptorPool(kudu::Sockaddr const&, std::__1::shared_ptr<kudu::rpc::AcceptorPool>*) /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:288:34 (libkrpc.so+0x16da89)
    #5 kudu::RpcServer::Bind() /apache/dev/git/kudu/src/kudu/server/rpc_server.cc:262:5 (libserver_process.so+0x1393fb)
    #6 kudu::server::ServerBase::Init() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1030:23 (libserver_process.so+0x149419)
    #7 kudu::kserver::KuduServer::Init() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:144:3 (libkserver.so+0x12e81)
    #8 kudu::tserver::TabletServer::Init() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:107:3 (libtserver.so+0x25c752)
    #9 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:106:3 (libtserver.so+0x31c79b)
    #10 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #11 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #12 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #13 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x35f97c)
    #14 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:431:3 (auto_leader_rebalancer-test+0x357913)
    #15 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #16 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #17 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #18 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #19 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #20 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #21 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #22 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #23 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #24 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #25 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

  Mutex M915773476609850416 is already destroyed.

  Thread T264 'diag-logger-238' (tid=23806, running) created by main thread at:
    #0 pthread_create /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (auto_leader_rebalancer-test+0x2d7bf5)
    #1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x40e0d8)
    #2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.h:147:12 (libmaster.so+0x2e29a9)
    #3 kudu::server::DiagnosticsLog::Start() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:144:14 (libserver_process.so+0x11d27d)
    #4 kudu::server::ServerBase::StartMetricsLogging() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1212:3 (libserver_process.so+0x14b6c4)
    #5 kudu::server::ServerBase::Start() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1352:3 (libserver_process.so+0x14bf64)
    #6 kudu::kserver::KuduServer::Start() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:179:22 (libkserver.so+0x13810)
    #7 kudu::tserver::TabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:161:3 (libtserver.so+0x25cf97)
    #8 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:107:3 (libtserver.so+0x31c7d9)
    #9 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #10 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #11 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #12 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x35f97c)
    #13 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:431:3 (auto_leader_rebalancer-test+0x357913)
    #14 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #15 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #16 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #17 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #18 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #19 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #20 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #21 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #22 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #23 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #24 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

SUMMARY: ThreadSanitizer: data race /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:96:7 in kudu::DiagnosticSocket::Close()
==================
I20260709 12:16:04.163328 30501 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.164021 30501 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.167807 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.168406 30501 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.168609 23925 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:16:04.169811 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:16:04.172945 25114 leader_election.cc:336] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.176107 30501 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:04.176617 30501 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.176994 30501 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.162460 30404 raft_consensus.cc:515] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.183673 30404 leader_election.cc:290] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.183925 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8341d6ef4764792b1bd6ff196e8af38" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:04.184599 23925 raft_consensus.cc:3060] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:04.185374 30500 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.185920 30500 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.188606 30517 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.189198 30517 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:16:04.192464 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.193293 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
W20260709 12:16:04.193619 25114 leader_election.cc:336] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.194139 30517 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.195317 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.196022 23854 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:16:04.197250 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:16:04.204226 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.205334 30517 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:04.205801 30517 raft_consensus.cc:493] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.206156 30517 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:04.208333 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:04.209189 30518 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:04.210119 30500 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.211679 30501 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.213007 23925 raft_consensus.cc:2468] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
W20260709 12:16:04.217252 25114 leader_election.cc:336] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
==================
WARNING: ThreadSanitizer: data race (pid=22629)
  Write of size 8 at 0x7ba0000004b0 by main thread:
    #0 close /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1679 (auto_leader_rebalancer-test+0x3067a3)
    #1 kudu::DiagnosticSocket::Close() /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:99:7 (libkudu_util.so+0x49fe1a)
    #2 kudu::rpc::AcceptorPool::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:212:3 (libkrpc.so+0x12ad33)
    #3 kudu::rpc::Messenger::ShutdownInternal(kudu::rpc::Messenger::ShutdownMode) /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:239:8 (libkrpc.so+0x16d257)
    #4 kudu::rpc::Messenger::Shutdown() /apache/dev/git/kudu/src/kudu/rpc/messenger.cc:206:3 (libkrpc.so+0x16d80b)
    #5 kudu::kserver::KuduServer::Shutdown() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:192:17 (libkserver.so+0x13865)
    #6 kudu::tserver::TabletServer::ShutdownImpl() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:195:17 (libtserver.so+0x25c2ca)
    #7 kudu::tserver::TabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x261a79)
    #8 kudu::tserver::MiniTabletServer::Shutdown() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x31c692)
    #9 kudu::cluster::InternalMiniCluster::ShutdownNodes(kudu::cluster::ClusterNodes) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:242:22 (libmini_cluster.so+0xd7a87)
    #10 kudu::cluster::MiniCluster::Shutdown() /apache/dev/git/kudu/src/kudu/mini-cluster/mini_cluster.h:84:5 (auto_leader_rebalancer-test+0x3758e4)
    #11 kudu::master::LeaderRebalancerTest::TearDown() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:116:17 (auto_leader_rebalancer-test+0x363142)
    #12 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #13 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #14 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2682:3 (libgtest.so.1.12.1+0x42a82)
    #15 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #16 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
I20260709 12:16:04.219098 30497 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
    #17 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #18 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #19 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #20 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #21 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #22 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

  Previous read of size 8 at 0x7ba0000004b0 by thread T264:
    [failed to restore the stack]

  Location is file descriptor 75 created by main thread at:
    #0 socket /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1638 (auto_leader_rebalancer-test+0x3094ff)
    #1 kudu::DiagnosticSocket::Init() /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:80:13 (libkudu_util.so+0x49ff2f)
    #2 kudu::rpc::AcceptorPool::Start(int) /apache/dev/git/kudu/src/kudu/rpc/acceptor_pool.cc:167:3 (libkrpc.so+0x12b265)
    #3 kudu::RpcServer::Start() /apache/dev/git/kudu/src/kudu/server/rpc_server.cc:297:5 (libserver_process.so+0x1398d9)
I20260709 12:16:04.219820 30497 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
    #4 kudu::server::ServerBase::Start() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1361:3 (libserver_process.so+0x14bff5)
    #5 kudu::kserver::KuduServer::Start() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:179:22 (libkserver.so+0x13810)
    #6 kudu::tserver::TabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:161:3 (libtserver.so+0x25cf97)
    #7 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:107:3 (libtserver.so+0x31c7d9)
    #8 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #9 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #10 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #11 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x35f97c)
    #12 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:431:3 (auto_leader_rebalancer-test+0x357913)
    #13 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #14 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #15 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #16 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #17 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #18 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #19 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #20 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #21 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #22 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #23 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

  Thread T264 'diag-logger-238' (tid=23806, running) created by main thread at:
    #0 pthread_create /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (auto_leader_rebalancer-test+0x2d7bf5)
    #1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x40e0d8)
    #2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /apache/dev/git/kudu/src/kudu/util/thread.h:147:12 (libmaster.so+0x2e29a9)
    #3 kudu::server::DiagnosticsLog::Start() /apache/dev/git/kudu/src/kudu/server/diagnostics_log.cc:144:14 (libserver_process.so+0x11d27d)
    #4 kudu::server::ServerBase::StartMetricsLogging() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1212:3 (libserver_process.so+0x14b6c4)
    #5 kudu::server::ServerBase::Start() /apache/dev/git/kudu/src/kudu/server/server_base.cc:1352:3 (libserver_process.so+0x14bf64)
    #6 kudu::kserver::KuduServer::Start() /apache/dev/git/kudu/src/kudu/kserver/kserver.cc:179:22 (libkserver.so+0x13810)
    #7 kudu::tserver::TabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/tablet_server.cc:161:3 (libtserver.so+0x25cf97)
    #8 kudu::tserver::MiniTabletServer::Start() /apache/dev/git/kudu/src/kudu/tserver/mini_tablet_server.cc:107:3 (libtserver.so+0x31c7d9)
    #9 kudu::cluster::InternalMiniCluster::AddTabletServer(kudu::HostPort const&) /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:234:3 (libmini_cluster.so+0xd7889)
    #10 kudu::cluster::InternalMiniCluster::AddTabletServer() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:215:10 (libmini_cluster.so+0xd73a5)
    #11 kudu::cluster::InternalMiniCluster::Start() /apache/dev/git/kudu/src/kudu/mini-cluster/internal_mini_cluster.cc:112:5 (libmini_cluster.so+0xd663a)
    #12 kudu::master::LeaderRebalancerTest::CreateAndStartCluster() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:104:22 (auto_leader_rebalancer-test+0x35f97c)
    #13 kudu::master::LeaderRebalancerTest_AddTserver_Test::TestBody() /apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:431:3 (auto_leader_rebalancer-test+0x357913)
    #14 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e0f)
    #15 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e0f)
    #16 testing::Test::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a11)
    #17 testing::TestInfo::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d28)
    #18 testing::TestSuite::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d04)
    #19 testing::internal::UnitTestImpl::RunAllTests() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597f4)
    #20 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65ccf)
    #21 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65ccf)
    #22 testing::UnitTest::Run() /apache/dev/git/kudu/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dac)
    #23 RUN_ALL_TESTS() /apache/dev/git/kudu/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (auto_leader_rebalancer-test+0x38839b)
    #24 main /apache/dev/git/kudu/src/kudu/util/test_main.cc:115:10 (auto_leader_rebalancer-test+0x386f9c)

SUMMARY: ThreadSanitizer: data race /apache/dev/git/kudu/src/kudu/util/net/diagnostic_socket.cc:99:7 in kudu::DiagnosticSocket::Close()
==================
I20260709 12:16:04.223676 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.224431 23934 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 17.
I20260709 12:16:04.225278 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:04.226106 23931 raft_consensus.cc:3060] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.227231 25113 leader_election.cc:304] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.228564 30404 raft_consensus.cc:2804] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:04.229127 30404 raft_consensus.cc:697] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.228408 30497 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.231146 30489 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.231958 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.232996 30516 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:04.233644 30516 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.234017 30516 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.231686 30404 consensus_queue.cc:237] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.231841 30489 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.239429 30489 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.241644 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:04.244872 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.255061 23931 raft_consensus.cc:2468] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
W20260709 12:16:04.256925 23825 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.257436 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.262208 30489 raft_consensus.cc:2749] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:04.265704 30516 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.273422 30489 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.274058 30489 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.260237 30517 raft_consensus.cc:515] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.259742 30501 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.276237 30516 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.276381 30489 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.277774 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e943fd8cd04e4a5e9678a4bdbd7b1449" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.278450 23860 raft_consensus.cc:3060] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:04.280320 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e76b9730b35144689e48d4d145fabb18" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:04.280943 23931 raft_consensus.cc:3060] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.291217 30516 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.291869 30516 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:04.287287 23860 raft_consensus.cc:2468] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:16:04.297580 23900 leader_election.cc:304] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.298812 30501 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.293052 30517 leader_election.cc:290] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.301424 25113 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.306176 30500 raft_consensus.cc:2804] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:04.306115 30501 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.307083 30500 raft_consensus.cc:697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:04.308158 30500 consensus_queue.cc:237] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:16:04.292224 23749 connection.cc:466] server connection from 127.0.0.1:43422 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43422 (request call id 3865) could send its response
I20260709 12:16:04.308864 30501 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.315326 23931 raft_consensus.cc:2468] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 18.
I20260709 12:16:04.306357 30516 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:04.317024 23825 connection.cc:573] client connection to 127.22.25.65:35817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:16:04.317322 23677 catalog_manager.cc:5697] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 17 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:04.320765 23899 connection.cc:573] client connection to 127.22.25.65:35817 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:16:04.327368 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:04.328217 25168 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 13.
I20260709 12:16:04.332077 30525 raft_consensus.cc:2804] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:04.355054 30521 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.355285 30497 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.355746 30521 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.355929 30497 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.357863 30497 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.357957 30521 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.334160 30525 raft_consensus.cc:697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.360185 30525 consensus_queue.cc:237] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:04.364878 25114 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:04.369850 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 77166.6us
I20260709 12:16:04.373303 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:16:04.375334 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 111362us
I20260709 12:16:04.376353 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.381145 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.382519 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.383221 23860 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:16:04.381909 23934 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 14.
I20260709 12:16:04.367182 30521 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.396471 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.396533 30521 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.397471 30404 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:04.397873 30404 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.398212 30404 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:16:04.399147 25114 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:04.438894 30535 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.439507 30535 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.441730 30535 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.445402 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.446192 23860 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
W20260709 12:16:04.444981 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 43822.3us
I20260709 12:16:04.449708 25119 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.448285 30404 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.432750 30526 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:04.456496 30534 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
W20260709 12:16:04.455271 25114 connection.cc:769] client connection to 127.22.25.65:35817 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:16:04.459609 30516 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.462954 30537 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.463176 30516 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.463630 30537 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.464287 30532 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:04.464938 30532 raft_consensus.cc:493] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.465435 30532 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.463001 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.467317 23934 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:16:04.469172 30404 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.454838 30526 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.454902 30497 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.474021 25113 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.471838 30526 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.479453 30404 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:04.457242 30534 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.481460 30404 raft_consensus.cc:493] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.488377 30404 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:04.493697 30534 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.496102 30516 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.498548 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.499334 23860 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:16:04.500797 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.500984 30521 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.482064 30497 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.501765 30537 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:04.493721 23825 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.502589 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.498215 23899 leader_election.cc:336] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.503121 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.510468 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.511451 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.512151 23854 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:16:04.513206 30497 leader_election.cc:290] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.514775 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.515413 23852 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 9.
I20260709 12:16:04.516685 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.517503 30537 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:04.517884 30537 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.518273 30537 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.525255 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.526338 30524 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:16:04.526765 30524 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.527141 30524 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:04.528725 30451 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:04.529168 30451 raft_consensus.cc:493] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.529506 30451 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:04.533450 23825 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.534777 30538 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.535392 30538 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:04.538223 30538 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.540009 30526 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.541209 23860 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 14: Already voted for candidate 438159d8de3c4027bdcf57eb81dedf86 in this term.
I20260709 12:16:04.543303 30526 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:04.541168 30451 raft_consensus.cc:515] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.553756 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.554607 23854 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 14.
I20260709 12:16:04.560973 30404 raft_consensus.cc:515] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.566781 30404 leader_election.cc:290] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.567934 30524 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.570279 30524 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.562434 30532 raft_consensus.cc:515] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.574102 30532 leader_election.cc:290] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:04.575362 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.576246 23825 leader_election.cc:336] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.577644 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.578228 23854 raft_consensus.cc:3060] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:04.581175 30451 leader_election.cc:290] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:04.583967 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 207694us
I20260709 12:16:04.584753 23824 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.585681 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc09548ba8eb4f01bf6d74a33d072dda" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.586265 23852 raft_consensus.cc:3060] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.589792 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.591536 30538 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:04.591971 30538 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.592320 30538 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:04.595526 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20edc48b2f884789ac3daead4b5c3102" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.597139 23860 raft_consensus.cc:3060] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:04.598452 23899 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.599563 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.599709 30537 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.600715 23676 catalog_manager.cc:5697] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 16, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 16 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.601890 30537 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.602411 30497 raft_consensus.cc:2804] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.603049 30451 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.602945 30497 raft_consensus.cc:697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:16:04.603780 30451 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.604323 30497 consensus_queue.cc:237] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.611109 23854 raft_consensus.cc:2468] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
I20260709 12:16:04.615054 23852 raft_consensus.cc:2468] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:16:04.616047 25119 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.616863 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.617542 23856 raft_consensus.cc:3060] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.625787 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b93ab9c24064567b717af205bc2ed2a" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:04.626582 23931 raft_consensus.cc:3060] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:04.627954 30520 raft_consensus.cc:2804] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:16:04.633448 30520 raft_consensus.cc:697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.634439 30520 consensus_queue.cc:237] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.636754 30526 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.637418 30526 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.639163 23931 raft_consensus.cc:2468] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
W20260709 12:16:04.638233 23899 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.640033 25119 leader_election.cc:304] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.639451 30526 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.626202 23860 raft_consensus.cc:2468] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
W20260709 12:16:04.642345 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 57681.3us
I20260709 12:16:04.647274 30404 raft_consensus.cc:2804] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:04.647576 23824 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:04.647856 30404 raft_consensus.cc:697] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.639814 23856 raft_consensus.cc:2468] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 18.
I20260709 12:16:04.651263 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.647533 23900 leader_election.cc:304] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:16:04.652105 23899 leader_election.cc:336] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.652667 30451 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.656816 30541 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:04.658735 30541 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:04.659147 30541 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.657698 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.659519 30541 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:04.659998 23852 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:16:04.661136 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.661965 30500 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:04.662343 30500 raft_consensus.cc:493] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.662524 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:04.662899 30500 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.657461 30534 raft_consensus.cc:2804] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:04.664834 30534 raft_consensus.cc:697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:04.665769 30534 consensus_queue.cc:237] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.667840 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.667711 25119 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.653710 30404 consensus_queue.cc:237] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.675884 30501 raft_consensus.cc:2804] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.676468 30501 raft_consensus.cc:697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.677322 30501 consensus_queue.cc:237] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 17.17, Last appended by leader: 17, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.685056 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:04.685983 23925 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 12.
I20260709 12:16:04.686982 25168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:16:04.688280 30451 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.688822 30451 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.690992 30451 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:04.674085 23825 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.704852 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.705763 23852 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:16:04.707069 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.710686 30451 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:16:04.700197 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 51196.5us
I20260709 12:16:04.698619 30538 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:04.755026 30541 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.710508 25168 raft_consensus.cc:2393] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 14: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:16:04.768582 30500 raft_consensus.cc:515] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.780798 23679 catalog_manager.cc:5697] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 17 to 18, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 18 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:04.786983 30451 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.790797 30451 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:04.802294 23931 raft_consensus.cc:1275] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:04.802469 23934 raft_consensus.cc:2393] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 438159d8de3c4027bdcf57eb81dedf86 in current term 18: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
I20260709 12:16:04.804828 25113 leader_election.cc:304] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:16:04.801550 25114 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.812803 30500 leader_election.cc:290] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.816068 30520 raft_consensus.cc:2804] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:04.816725 30520 raft_consensus.cc:697] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:04.817703 30520 consensus_queue.cc:237] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.823252 30541 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:04.824834 30532 consensus_queue.cc:1048] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:04.825307 30538 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.823326 30525 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.827508 30525 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.836253 30525 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.841626 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:04.842374 23852 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
I20260709 12:16:04.857448 30451 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:04.860481 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.861301 23856 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:04.862118 25119 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 
I20260709 12:16:04.862826 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2844521776744308787188de9041ebd" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.863536 23852 raft_consensus.cc:3060] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.864666 30404 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:04.865084 30404 raft_consensus.cc:493] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0ee4ab0747914e7abc642312517b5e25)
I20260709 12:16:04.865409 30404 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:04.866115 25114 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.867048 25114 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.867532 25114 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:04.878037 23856 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:16:04.880398 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.884670 23852 raft_consensus.cc:2468] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 18.
I20260709 12:16:04.885699 30501 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:16:04.885720 25114 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.896350 25114 leader_election.cc:336] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.897111 25114 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.897764 25114 leader_election.cc:336] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.898056 30500 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:16:04.898391 25114 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.898646 30500 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:04.886771 23900 leader_election.cc:304] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:04.899732 30500 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:04.900262 30517 raft_consensus.cc:2804] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.900887 30517 raft_consensus.cc:697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:04.901739 30517 consensus_queue.cc:237] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:16:04.914096 23825 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.917400 23824 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.930079 30550 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:16:04.916601 30510 tablet_copy_client.cc:1131] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.932615 25114 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:04.933491 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.921903 30404 raft_consensus.cc:515] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:04.940078 30497 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:16:04.948436 30404 leader_election.cc:290] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.952056 23852 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "055432f1640d4d5a8a4ec2355e0b89ac" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.952719 23852 raft_consensus.cc:3060] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:04.969640 25114 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:04.970722 25114 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:04.983908 25114 leader_election.cc:336] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:04.988582 23856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:04.989287 23856 raft_consensus.cc:3060] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:04.990125 23860 raft_consensus.cc:1275] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:16:04.973017 30451 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:04.971700 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2366a1e0cb342ee891c776affbfbca3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:04.992228 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:04.992698 23934 raft_consensus.cc:3060] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:04.997906 23852 raft_consensus.cc:2468] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:16:05.006362 23824 leader_election.cc:304] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:05.007099 30523 consensus_queue.cc:1048] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:16:05.007819 30516 raft_consensus.cc:2749] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:05.010093 25119 leader_election.cc:304] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:05.011487 30520 raft_consensus.cc:2804] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:05.010088 23934 raft_consensus.cc:2468] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 15.
I20260709 12:16:05.012876 30520 raft_consensus.cc:697] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:05.016789 30520 consensus_queue.cc:237] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:05.024281 23856 raft_consensus.cc:2468] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:16:05.026706 23900 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:05.027000 30497 raft_consensus.cc:2764] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:16:05.027911 30523 raft_consensus.cc:2804] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:16:05.029302 23824 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:05.031088 30541 raft_consensus.cc:2804] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:05.031666 30541 raft_consensus.cc:697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:16:05.032634 30541 consensus_queue.cc:237] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:05.036506 23934 raft_consensus.cc:1275] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:16:05.037321 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 100371us
I20260709 12:16:05.028458 30523 raft_consensus.cc:697] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:05.095768 30523 consensus_queue.cc:237] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:05.117098 23676 catalog_manager.cc:5697] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:05.126279 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 71371.6us
I20260709 12:16:05.119040 23676 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 14 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 15 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:05.131628 23676 catalog_manager.cc:5697] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 17 to 18, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 18 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:05.139197 23825 consensus_peers.cc:597] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:05.109277 30539 consensus_queue.cc:1048] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 12:16:05.147071 23899 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:05.148481 25114 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.149116 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.152853 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.153436 25114 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.158641 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.159329 23852 raft_consensus.cc:1275] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 17 index: 17. Preceding OpId from leader: term: 18 index: 18. (index mismatch)
I20260709 12:16:05.160822 23677 catalog_manager.cc:5697] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 14 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 15 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:05.167829 30539 consensus_queue.cc:1048] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:05.172115 23679 catalog_manager.cc:5697] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 14, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.175782 23679 catalog_manager.cc:5697] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 9 to 10, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 10 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.177507 23679 catalog_manager.cc:5697] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 16 to 17, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 17 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.179032 23679 catalog_manager.cc:5697] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 14, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:05.182994 23899 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:05.180557 23679 catalog_manager.cc:5697] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 15 to 16, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 16 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.185395 23679 catalog_manager.cc:5697] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 17 to 18, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 18 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:05.184281 23899 leader_election.cc:336] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:05.191509 23899 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:05.192628 23899 leader_election.cc:336] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:05.207846 23899 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.208930 23899 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.213883 23926 raft_consensus.cc:1275] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:05.216265 23925 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:50860
I20260709 12:16:05.217051 23925 raft_consensus.cc:606] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership
W20260709 12:16:05.218644 23899 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:05.219405 23854 raft_consensus.cc:1275] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:16:05.223201 23825 consensus_peers.cc:597] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.225347 30501 consensus_queue.cc:1048] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:16:05.241641 23899 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.243081 23899 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.249576 23925 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:50860
I20260709 12:16:05.250186 23925 raft_consensus.cc:606] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership
I20260709 12:16:05.254788 30551 consensus_queue.cc:1048] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:16:05.271768 30565 log.cc:927] Time spent T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:16:05.284322 23860 raft_consensus.cc:1275] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:05.288220 23860 raft_consensus.cc:1275] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:16:05.290002 30525 consensus_queue.cc:1048] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:05.290390 30523 consensus_queue.cc:1048] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:16:05.297483 25114 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.315054 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.319525 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.340111 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.342711 23668 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 91250.5us
I20260709 12:16:05.344745 23931 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:50860
I20260709 12:16:05.345283 23931 raft_consensus.cc:606] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Received request to transfer leadership
W20260709 12:16:05.350088 23731 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:16:05.350503 23731 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for c395cd7fe30b41869bce6dac359f2cf8 already in progress
W20260709 12:16:05.350777 23731 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for c395cd7fe30b41869bce6dac359f2cf8 already in progress
W20260709 12:16:05.354676 25114 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.356731 23825 consensus_peers.cc:597] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.357086 23825 consensus_peers.cc:597] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.357328 23825 consensus_peers.cc:597] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.368626 23899 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.368531 23676 catalog_manager.cc:5697] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 12 to 13, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 13 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:05.378856 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.386757 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.395228 23934 raft_consensus.cc:1275] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:16:05.398030 30520 consensus_queue.cc:1048] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:05.416955 23860 raft_consensus.cc:1275] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:05.423545 30554 consensus_queue.cc:1048] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:05.425770 23856 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:16:05.431077 30556 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:16:05.432233 23825 consensus_peers.cc:597] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.441504 23899 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.441896 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.460122 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.478356 22629 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 23806)
W20260709 12:16:05.488008 23899 consensus_peers.cc:597] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:05.488658 25174 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEARNER]: Advancing to term 15
W20260709 12:16:05.497963 23899 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.512228 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.515661 23899 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.521886 30554 raft_consensus.cc:993] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: : Instructing follower 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:16:05.522456 30554 raft_consensus.cc:1081] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Signalling peer 438159d8de3c4027bdcf57eb81dedf86 to start an election
I20260709 12:16:05.525179 23856 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8"
dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
 from {username='slave'} at 127.0.0.1:56656
I20260709 12:16:05.525851 23856 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:05.526263 23856 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:05.517944 25174 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:16:05.532234 23854 raft_consensus.cc:1275] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:16:05.533869 30533 consensus_queue.cc:1048] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:16:05.540062 30556 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:05.540383 23856 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:05.544682 23860 raft_consensus.cc:1240] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Rejecting Update request from peer 9e0bf56d58c5480a8d088c85566558da for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:05.546774 23925 raft_consensus.cc:1275] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:16:05.547055 23825 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:05.551997 30541 consensus_queue.cc:1048] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:16:05.555351 30533 consensus_queue.cc:1059] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:16:05.558176 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.560766 23856 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:05.560945 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:05.565651 23934 raft_consensus.cc:3055] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:05.566018 23934 raft_consensus.cc:740] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:05.566797 23934 consensus_queue.cc:260] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:05.568256 23934 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:16:05.575070 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.585852 23825 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.586110 23934 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 438159d8de3c4027bdcf57eb81dedf86 in term 16.
W20260709 12:16:05.586779 30500 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:16:05.593914 23825 consensus_peers.cc:597] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.596957 23899 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.598155 23825 consensus_peers.cc:597] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.603883 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.651835 25114 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:05.657224 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 46748us
I20260709 12:16:05.659962 23824 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:05.661417 30551 raft_consensus.cc:2804] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:05.661928 30551 raft_consensus.cc:697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Becoming Leader. State: Replica: 438159d8de3c4027bdcf57eb81dedf86, State: Running, Role: LEADER
I20260709 12:16:05.662860 30551 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:05.688680 23676 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: term changed from 15 to 16, leader changed from 9e0bf56d58c5480a8d088c85566558da (127.22.25.67) to 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66). New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:05.694067 23856 raft_consensus.cc:1275] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:16:05.695228 23899 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:05.701712 30520 consensus_queue.cc:1048] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:16:05.709690 23825 consensus_peers.cc:597] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:05.749770 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:06.147315 25175 raft_consensus.cc:3060] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEARNER]: Advancing to term 16
I20260709 12:16:06.156939 25175 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:16:06.172060 30526 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:06.184180 30563 raft_consensus.cc:1064] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:16:06.184689 30563 raft_consensus.cc:1068] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: Unable to promote non-voter 05679ef5e2b74861aad2a08b6a05e276: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:16:06.247252 23925 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:16:06.257719 30541 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:16:06.275310 23825 consensus_peers.cc:597] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:06.352260 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 61206.6us
W20260709 12:16:06.478968 22629 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 23806)
I20260709 12:16:06.518002 23854 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
W20260709 12:16:06.653730 23824 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683): Couldn't send request to peer 05679ef5e2b74861aad2a08b6a05e276. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 861b3b9e09c14ef99794d4c8d50d59ad. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:06.702140 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 48654.1us
I20260709 12:16:06.718252 30541 raft_consensus.cc:1064] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
I20260709 12:16:06.724592 30541 consensus_queue.cc:237] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:06.732983 25174 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:06.735008 23925 raft_consensus.cc:1275] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:16:06.737794 23825 consensus_peers.cc:597] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:06.739156 30601 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:06.739653 23925 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:16:06.744704 23825 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:06.746305 30604 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:06.752103 30541 consensus_queue.cc:1048] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:06.775488 30601 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 9 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:06.777462 25175 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 9 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:06.798633 23677 catalog_manager.cc:5697] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: config changed from index 9 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:06.811481 30608 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.813100 30608 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:06.819553 23931 raft_consensus.cc:2955] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
W20260709 12:16:06.822127 25114 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:06.820894 30604 raft_consensus.cc:2955] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:16:06.838626 23854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86" is_pre_election: true
I20260709 12:16:06.839385 23854 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:16:06.840777 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:06.841676 30607 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:06.842221 30607 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.842578 30607 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:06.851434 30608 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:06.862310 23676 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da reported cstate change: config changed from index -1 to 11, NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) added. New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } } }
I20260709 12:16:06.869031 23931 raft_consensus.cc:2955] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 9 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:06.862941 30607 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:06.881572 25114 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:06.892993 30607 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Requested vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:06.893535 23860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "438159d8de3c4027bdcf57eb81dedf86"
I20260709 12:16:06.894146 23860 raft_consensus.cc:3060] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:06.900914 23860 raft_consensus.cc:2468] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:16:06.902129 25119 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:06.903811 30607 raft_consensus.cc:2804] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:06.904377 30607 raft_consensus.cc:697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:06.905416 30607 consensus_queue.cc:237] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:06.984138 23676 catalog_manager.cc:5697] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 15, leader changed from 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 15 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:07.028015 25114 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:07.188571 30617 ts_tablet_manager.cc:933] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Initiating tablet copy from peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:07.216039 30617 tablet_copy_client.cc:323] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Beginning tablet copy session from remote peer at address 127.22.25.66:44897
I20260709 12:16:07.220852 23870 tablet_copy_service.cc:140] P 438159d8de3c4027bdcf57eb81dedf86: Received BeginTabletCopySession request for tablet 861b3b9e09c14ef99794d4c8d50d59ad from peer 05679ef5e2b74861aad2a08b6a05e276 ({username='slave'} at 127.0.0.1:46448)
I20260709 12:16:07.221429 23870 tablet_copy_service.cc:161] P 438159d8de3c4027bdcf57eb81dedf86: Beginning new tablet copy session on tablet 861b3b9e09c14ef99794d4c8d50d59ad from peer 05679ef5e2b74861aad2a08b6a05e276 at {username='slave'} at 127.0.0.1:46448: session id = 05679ef5e2b74861aad2a08b6a05e276-861b3b9e09c14ef99794d4c8d50d59ad
I20260709 12:16:07.229871 23870 tablet_copy_source_session.cc:215] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:07.242743 30617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861b3b9e09c14ef99794d4c8d50d59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:07.280818 30617 tablet_copy_client.cc:806] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:07.281646 30617 tablet_copy_client.cc:670] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:07.290007 30617 tablet_copy_client.cc:538] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:07.308723 30617 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap starting.
W20260709 12:16:07.332237 25114 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:07.407351 30617 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:07.420446 30617 tablet_bootstrap.cc:492] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Bootstrap complete.
I20260709 12:16:07.421329 30617 ts_tablet_manager.cc:1403] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Time spent bootstrapping tablet: real 0.113s	user 0.046s	sys 0.010s
I20260709 12:16:07.424579 30617 raft_consensus.cc:359] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:07.425570 30617 raft_consensus.cc:740] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Initialized, Role: LEARNER
I20260709 12:16:07.426345 30617 consensus_queue.cc:260] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: NON_VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: true } }
I20260709 12:16:07.428647 30617 ts_tablet_manager.cc:1434] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:07.447209 23856 raft_consensus.cc:1275] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:16:07.449608 23870 tablet_copy_service.cc:342] P 438159d8de3c4027bdcf57eb81dedf86: Request end of tablet copy session 05679ef5e2b74861aad2a08b6a05e276-861b3b9e09c14ef99794d4c8d50d59ad received from {username='slave'} at 127.0.0.1:46448
I20260709 12:16:07.450127 23870 tablet_copy_service.cc:434] P 438159d8de3c4027bdcf57eb81dedf86: ending tablet copy session 05679ef5e2b74861aad2a08b6a05e276-861b3b9e09c14ef99794d4c8d50d59ad on tablet 861b3b9e09c14ef99794d4c8d50d59ad with peer 05679ef5e2b74861aad2a08b6a05e276
I20260709 12:16:07.450383 30608 consensus_queue.cc:1048] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:16:07.479615 22629 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 23806)
W20260709 12:16:07.495095 25114 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:07.576867 23825 consensus_peers.cc:597] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.578224 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.579643 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.585008 23899 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.605165 25114 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.606438 23899 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.652819 23899 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.661502 25114 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.666433 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.677811 25114 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:07.704732 25168 raft_consensus.cc:1217] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
W20260709 12:16:07.732560 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.734356 23899 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.734901 23899 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.743536 25114 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.758738 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.758272 23825 consensus_peers.cc:597] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.767885 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.815752 23825 consensus_peers.cc:597] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.836257 23899 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.840119 23899 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.865836 23825 consensus_peers.cc:597] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.915823 23899 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.935808 23825 consensus_peers.cc:597] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.969381 23825 consensus_peers.cc:597] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:07.985610 23825 consensus_peers.cc:597] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:07.991143 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.014866 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.064198 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.087189 23899 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:08.087270 23825 consensus_peers.cc:597] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.092396 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.108845 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.159132 23899 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.195359 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 90965.5us
W20260709 12:16:08.215027 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.230196 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.230773 25114 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:08.231169 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.241207 23825 consensus_peers.cc:597] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:08.242009 25114 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111) [suppressed 429 similar messages]
W20260709 12:16:08.231256 23899 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:08.292346 30623 raft_consensus.cc:1064] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: attempting to promote NON_VOTER 05679ef5e2b74861aad2a08b6a05e276 to VOTER
W20260709 12:16:08.293874 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:08.295217 30623 consensus_queue.cc:237] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:08.358935 23825 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.362857 25175 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEARNER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:08.364151 23925 raft_consensus.cc:1275] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Refusing update from remote peer 438159d8de3c4027bdcf57eb81dedf86: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:08.367271 30601 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:08.370483 30601 consensus_queue.cc:1048] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:08.416213 30630 raft_consensus.cc:2955] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:08.421424 23925 raft_consensus.cc:2955] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:08.463874 25175 raft_consensus.cc:2955] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } } }
I20260709 12:16:08.472575 23676 catalog_manager.cc:5697] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 reported cstate change: config changed from index 11 to 12, 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "438159d8de3c4027bdcf57eb81dedf86" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:08.480118 22629 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 23806)
W20260709 12:16:09.401249 23825 consensus_peers.cc:597] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:09.646296 25114 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:09.721567 23898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.68:43683, user_credentials={real_user=slave}} blocked reactor thread for 48615.3us
W20260709 12:16:09.870702 23824 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 126272us
W20260709 12:16:09.875718 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 119910us
I20260709 12:16:10.211228 22629 tablet_server.cc:196] TabletServer@127.22.25.65:0 shutdown complete.
W20260709 12:16:10.316663 23825 consensus_peers.cc:597] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.387806 22629 tablet_server.cc:179] TabletServer@127.22.25.66:0 shutting down...
W20260709 12:16:10.515772 23899 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.049s	sys 0.092s
W20260709 12:16:10.516131 23899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.049s	sys 0.092s
W20260709 12:16:10.526605 25114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.021s	sys 0.017s
W20260709 12:16:10.527011 25114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.022s	sys 0.017s
W20260709 12:16:10.534456 23900 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.014s
W20260709 12:16:10.534859 23900 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.004s	sys 0.014s
W20260709 12:16:10.535120 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 574066us
W20260709 12:16:10.536689 23825 consensus_peers.cc:597] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.588203 23825 consensus_peers.cc:597] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:10.611901 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.008s	sys 0.022s
W20260709 12:16:10.612322 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.008s	sys 0.023s
W20260709 12:16:10.615818 25119 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.620185 23900 consensus_peers.cc:597] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.622129 23900 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.624212 23900 consensus_peers.cc:597] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.616693 25119 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.625407 23900 consensus_peers.cc:597] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.632486 25119 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.636108 23825 consensus_peers.cc:597] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.633281 23900 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.638571 25119 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.643214 25119 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.654187 23900 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.655030 23900 consensus_peers.cc:597] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.679248 23825 consensus_peers.cc:597] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.681823 25114 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:10.691793 25119 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.702870 25119 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.703886 25119 consensus_peers.cc:597] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.705714 23825 consensus_peers.cc:597] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.698832 23900 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.699771 25114 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:10.710842 25114 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:10.711401 25114 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:10.709475 25119 consensus_peers.cc:597] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.715761 25119 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.731045 23900 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.719201 25119 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.731809 23900 consensus_peers.cc:597] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.734232 30642 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:10.734819 30642 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.736922 30642 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.737892 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:10.745985 23900 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.746632 23900 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.757958 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8368f2e695204683affed6a9bd84ba96" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:10.770052 30630 raft_consensus.cc:493] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.771979 30630 raft_consensus.cc:515] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.774118 30630 leader_election.cc:290] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.774978 23824 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:10.775545 30661 raft_consensus.cc:493] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.779300 30667 raft_consensus.cc:2749] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:16:10.779868 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:10.781358 30671 raft_consensus.cc:493] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.780459 25177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f2d1f3ed104c1e99f82ebe04843248" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:10.779500 30661 raft_consensus.cc:515] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.781898 30671 raft_consensus.cc:515] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.784137 30661 leader_election.cc:290] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.784157 30671 leader_election.cc:290] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.786541 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "349b687db5c44196bafc207c26ca3679" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:10.789855 23825 consensus_peers.cc:597] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:10.792438 30666 raft_consensus.cc:493] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:10.792591 30665 raft_consensus.cc:493] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:10.793198 30665 raft_consensus.cc:515] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:10.795514 30665 leader_election.cc:290] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:10.793108 30666 raft_consensus.cc:515] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:10.802062 30671 raft_consensus.cc:493] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.805471 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:10.802686 30666 leader_election.cc:290] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:10.802629 30671 raft_consensus.cc:515] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.811897 30671 leader_election.cc:290] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:16:10.812479 23825 leader_election.cc:336] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:10.813040 23825 leader_election.cc:336] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.807331 23824 leader_election.cc:304] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:10.816438 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233089f4b82a4b1996ad9129dc392819" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:10.819237 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e60a1c6520545729d27474d8f237ce5" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:10.819653 30671 raft_consensus.cc:2749] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:10.820963 23926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3c66ee31004577b517e2c4322696b6" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:16:10.822047 23825 leader_election.cc:336] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.822922 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a979dd022fc24afd8765a3cdb5f7fab3" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:10.823011 23824 leader_election.cc:304] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:10.828217 30671 raft_consensus.cc:2749] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:10.828980 23824 leader_election.cc:304] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:10.829857 30671 raft_consensus.cc:2749] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:16:10.830423 23824 leader_election.cc:304] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:10.831223 30671 raft_consensus.cc:2749] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:16:10.836778 23824 leader_election.cc:304] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:10.837603 30666 raft_consensus.cc:2749] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:10.838119 30671 raft_consensus.cc:493] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:10.838657 30671 raft_consensus.cc:515] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:10.840965 30671 leader_election.cc:290] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:10.843986 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34e5e0c10245477d9010ece72d08ae51" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:16:10.847733 23825 leader_election.cc:336] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.848922 23824 leader_election.cc:304] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:10.849687 23899 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.850309 23899 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:10.850683 23899 leader_election.cc:336] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.851051 23899 leader_election.cc:304] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:10.851835 30661 raft_consensus.cc:2749] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:10.849666 30671 raft_consensus.cc:2749] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:10.854116 30666 raft_consensus.cc:493] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.854626 30666 raft_consensus.cc:515] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.856901 30666 leader_election.cc:290] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.864616 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "643746a7a11243df94746cf1081fc078" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:10.866051 23825 leader_election.cc:336] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.868095 23824 leader_election.cc:304] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:10.868872 30671 raft_consensus.cc:2749] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:10.869442 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:10.872269 22629 tablet_replica.cc:333] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.873126 22629 raft_consensus.cc:2243] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.874239 22629 raft_consensus.cc:2272] T 053977846bfa4adf9703ea669a81f41d P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.876956 22629 tablet_replica.cc:333] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.877799 22629 raft_consensus.cc:2243] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.878813 22629 raft_consensus.cc:2272] T b7caafee6779404494f8df62d888c5ef P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.881606 22629 tablet_replica.cc:333] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.882184 22629 raft_consensus.cc:2243] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.883045 22629 raft_consensus.cc:2272] T e2366a1e0cb342ee891c776affbfbca3 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.885440 22629 tablet_replica.cc:333] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.886209 22629 raft_consensus.cc:2243] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.887279 22629 raft_consensus.cc:2272] T d79b6f8bf5274da7a3f667b9988c5a84 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.889796 22629 tablet_replica.cc:333] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.890586 22629 raft_consensus.cc:2243] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.891675 22629 raft_consensus.cc:2272] T e76b9730b35144689e48d4d145fabb18 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.894119 22629 tablet_replica.cc:333] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.895895 30666 raft_consensus.cc:493] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.896430 30666 raft_consensus.cc:515] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.898564 30666 leader_election.cc:290] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:16:10.900002 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.901228 22629 raft_consensus.cc:2243] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.902484 22629 raft_consensus.cc:2272] T a1b618b31b2e427b8c073b2c24771903 P 438159d8de3c4027bdcf57eb81dedf86 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.902688 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20fc63afa03b45c0bf3f58aae82f3e3d" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 18 candidate_status { last_received { term: 17 index: 17 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:10.906213 23825 leader_election.cc:336] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:10.906677 23825 leader_election.cc:304] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:10.907616 30666 raft_consensus.cc:2749] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:10.912220 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.917863 22629 tablet_replica.cc:333] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
W20260709 12:16:10.922564 23825 consensus_peers.cc:597] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.922762 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:10.923384 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.924624 22629 raft_consensus.cc:2243] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.925946 22629 raft_consensus.cc:2272] T 71cb2ce294574daa8d7244ae252e8aa6 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.930625 30666 raft_consensus.cc:493] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:10.931169 30666 raft_consensus.cc:515] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:10.933504 30666 leader_election.cc:290] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:10.933198 22629 tablet_replica.cc:333] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.935132 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d6300a13a3401283a46bf3d2a07e82" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:10.935937 22629 raft_consensus.cc:2243] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.937227 22629 raft_consensus.cc:2272] T 5038ac294ceb4758b5b5de70f2469d5d P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.943120 23825 leader_election.cc:336] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:10.943216 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.943813 23825 leader_election.cc:304] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:10.944960 22629 tablet_replica.cc:333] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.945679 30666 raft_consensus.cc:2749] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:10.945889 22629 raft_consensus.cc:2243] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.946595 22629 raft_consensus.cc:2272] T 1567865ea02a4884996ae9d8beb9fa29 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.949357 22629 tablet_replica.cc:333] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.951010 22629 raft_consensus.cc:2243] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.951685 22629 raft_consensus.cc:2272] T 34e5e0c10245477d9010ece72d08ae51 P 438159d8de3c4027bdcf57eb81dedf86 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.957505 22629 tablet_replica.cc:333] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.960558 22629 raft_consensus.cc:2243] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.961208 22629 raft_consensus.cc:2272] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.963800 22629 tablet_replica.cc:333] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.964578 22629 raft_consensus.cc:2243] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.965186 22629 raft_consensus.cc:2272] T 4e60a1c6520545729d27474d8f237ce5 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.968243 22629 tablet_replica.cc:333] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.969058 22629 raft_consensus.cc:2243] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.969681 22629 raft_consensus.cc:2272] T d6d3d4f634284532bbf6ad7d4ac19214 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.972512 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:10.977240 22629 tablet_replica.cc:333] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.978209 22629 raft_consensus.cc:2243] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.978860 22629 raft_consensus.cc:2272] T 233089f4b82a4b1996ad9129dc392819 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.981616 22629 tablet_replica.cc:333] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.982417 22629 raft_consensus.cc:2243] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.983549 22629 raft_consensus.cc:2272] T c8b83386d2fa4ed2a212fbe8873239ea P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.986236 22629 tablet_replica.cc:333] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.987076 22629 raft_consensus.cc:2243] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.987702 22629 raft_consensus.cc:2272] T 34d6300a13a3401283a46bf3d2a07e82 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.990638 22629 tablet_replica.cc:333] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:10.991431 22629 raft_consensus.cc:2243] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.992149 22629 raft_consensus.cc:2272] T 3eeb67716b0e4f49aa1a4bd5815f684b P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.995339 23825 consensus_peers.cc:597] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:11.005139 22629 tablet_replica.cc:333] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.006443 22629 raft_consensus.cc:2243] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.015825 22629 raft_consensus.cc:2272] T 20edc48b2f884789ac3daead4b5c3102 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.018615 22629 tablet_replica.cc:333] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.019481 22629 raft_consensus.cc:2243] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:11.020133 25114 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:11.031898 22629 raft_consensus.cc:2272] T f5bb29269e8a4aa1b79f91e4ed687053 P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.034766 22629 tablet_replica.cc:333] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.035687 22629 raft_consensus.cc:2243] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.036347 22629 raft_consensus.cc:2272] T 526cc198da134bb590358ed73c23d59e P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.037505 30666 raft_consensus.cc:493] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:11.038033 30666 raft_consensus.cc:515] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:11.038818 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.039193 22629 tablet_replica.cc:333] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.043111 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4369a8cdaaa45beac1359775fa54781" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 19 candidate_status { last_received { term: 18 index: 18 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
W20260709 12:16:11.047034 23899 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:11.050909 30666 leader_election.cc:290] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:11.060108 22629 raft_consensus.cc:2243] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.060879 22629 raft_consensus.cc:2272] T fc09548ba8eb4f01bf6d74a33d072dda P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.062309 23825 leader_election.cc:336] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:11.063359 22629 tablet_replica.cc:333] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.064232 22629 raft_consensus.cc:2243] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.064807 22629 raft_consensus.cc:2272] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.062915 23825 leader_election.cc:304] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
W20260709 12:16:11.067046 23825 raft_consensus.cc:2686] T c4369a8cdaaa45beac1359775fa54781 P 438159d8de3c4027bdcf57eb81dedf86: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:11.068354 23900 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:11.069344 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.075724 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.067209 22629 tablet_replica.cc:333] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.084684 22629 raft_consensus.cc:2243] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.085585 22629 raft_consensus.cc:2272] T 24dc0370e122479081184511cb1224f1 P 438159d8de3c4027bdcf57eb81dedf86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.088315 22629 tablet_replica.cc:333] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.089102 22629 raft_consensus.cc:2243] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.089681 22629 raft_consensus.cc:2272] T c2844521776744308787188de9041ebd P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.090624 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.092125 22629 tablet_replica.cc:333] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.093130 22629 raft_consensus.cc:2243] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.094709 22629 raft_consensus.cc:2272] T c395cd7fe30b41869bce6dac359f2cf8 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.097765 22629 tablet_replica.cc:333] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.098546 22629 raft_consensus.cc:2243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.099560 22629 raft_consensus.cc:2272] T 20fc63afa03b45c0bf3f58aae82f3e3d P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.102087 22629 tablet_replica.cc:333] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.102877 22629 raft_consensus.cc:2243] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.103447 22629 raft_consensus.cc:2272] T 8368f2e695204683affed6a9bd84ba96 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.106633 22629 tablet_replica.cc:333] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.107410 22629 raft_consensus.cc:2243] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.108623 22629 raft_consensus.cc:2272] T c7203e0ca92840388d931c03974d32ff P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.111510 22629 tablet_replica.cc:333] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.112389 22629 raft_consensus.cc:2243] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.113088 22629 raft_consensus.cc:2272] T 8f333e0a484248ce88f0905e18500838 P 438159d8de3c4027bdcf57eb81dedf86 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.116739 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800193e80 after lost signal to thread 22632
W20260709 12:16:11.118013 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080014d6c0 after lost signal to thread 23717
W20260709 12:16:11.119427 23881 debug-util.cc:398] Leaking SignalData structure 0x7b080005c900 after lost signal to thread 23884
I20260709 12:16:11.120366 30666 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:16:11.120774 23881 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8b80 after lost signal to thread 23958
I20260709 12:16:11.120944 30666 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:11.121665 22634 debug-util.cc:398] Leaking SignalData structure 0x7b080000fc40 after lost signal to thread 23899
W20260709 12:16:11.122076 23881 debug-util.cc:398] Leaking SignalData structure 0x7b0800115f00 after lost signal to thread 25207
I20260709 12:16:11.123900 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:11.120796 22629 tablet_replica.cc:333] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.126745 22629 raft_consensus.cc:2243] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.128257 22629 raft_consensus.cc:2272] T 861b3b9e09c14ef99794d4c8d50d59ad P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.141701 30666 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:11.144126 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8a875cf4b4acea62c1086994499e4" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:11.150677 23824 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:11.151923 30666 raft_consensus.cc:2749] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:11.148871 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.145881 25119 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:11.175148 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.177412 22629 tablet_replica.cc:333] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.180099 22629 raft_consensus.cc:2243] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 11 LEADER]: Raft consensus shutting down.
W20260709 12:16:11.183120 25119 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:11.190361 22629 raft_consensus.cc:2272] T d9f2f570298740c691fa19d449b3fa4f P 438159d8de3c4027bdcf57eb81dedf86 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.190954 25114 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:11.204572 22629 tablet_replica.cc:333] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.205461 22629 raft_consensus.cc:2243] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.206107 22629 raft_consensus.cc:2272] T a979dd022fc24afd8765a3cdb5f7fab3 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.209339 22629 tablet_replica.cc:333] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.210110 22629 raft_consensus.cc:2243] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.211297 22629 raft_consensus.cc:2272] T 0b23f5ca4f084b8e96cc4a3099e99074 P 438159d8de3c4027bdcf57eb81dedf86 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.216774 30666 raft_consensus.cc:493] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 05679ef5e2b74861aad2a08b6a05e276)
I20260709 12:16:11.217329 30666 raft_consensus.cc:515] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:11.220789 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102493ff11494b009f79963bb57be511" candidate_uuid: "438159d8de3c4027bdcf57eb81dedf86" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:11.227105 30666 leader_election.cc:290] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
W20260709 12:16:11.228833 23825 leader_election.cc:336] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:11.229334 23825 leader_election.cc:304] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 438159d8de3c4027bdcf57eb81dedf86; no voters: 05679ef5e2b74861aad2a08b6a05e276, 0ee4ab0747914e7abc642312517b5e25
I20260709 12:16:11.230473 30666 raft_consensus.cc:2749] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:16:11.231472 23899 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.232117 23899 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:11.238034 23899 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.242421 22629 tablet_replica.cc:333] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.243340 22629 raft_consensus.cc:2243] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.244028 22629 raft_consensus.cc:2272] T 643746a7a11243df94746cf1081fc078 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.253190 22629 tablet_replica.cc:333] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.254036 22629 raft_consensus.cc:2243] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.255172 22629 raft_consensus.cc:2272] T 78f1c4dc751a4f4a93a446a785edda51 P 438159d8de3c4027bdcf57eb81dedf86 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.256385 23899 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.268232 23899 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.268507 22629 tablet_replica.cc:333] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.269336 22629 raft_consensus.cc:2243] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.269928 22629 raft_consensus.cc:2272] T ed3c66ee31004577b517e2c4322696b6 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.272639 22629 tablet_replica.cc:333] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.273392 22629 raft_consensus.cc:2243] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.274958 22629 raft_consensus.cc:2272] T 10c8a875cf4b4acea62c1086994499e4 P 438159d8de3c4027bdcf57eb81dedf86 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.278333 22629 tablet_replica.cc:333] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.279114 22629 raft_consensus.cc:2243] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.280345 22629 raft_consensus.cc:2272] T 98e5bc86804f4f979f58779caa696814 P 438159d8de3c4027bdcf57eb81dedf86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.283787 22629 tablet_replica.cc:333] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.284526 22629 raft_consensus.cc:2243] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.293941 22629 raft_consensus.cc:2272] T b54dd3a3d61243bea2dae318ccea3958 P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.296687 22629 tablet_replica.cc:333] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.297438 22629 raft_consensus.cc:2243] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.298039 22629 raft_consensus.cc:2272] T de3c70defd864769b26b52b7f36548cd P 438159d8de3c4027bdcf57eb81dedf86 [term 9 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.304919 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:11.316946 22629 tablet_replica.cc:333] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.317915 22629 raft_consensus.cc:2243] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.318526 22629 raft_consensus.cc:2272] T 102493ff11494b009f79963bb57be511 P 438159d8de3c4027bdcf57eb81dedf86 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.321060 22629 tablet_replica.cc:333] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.321738 22629 raft_consensus.cc:2243] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.322307 22629 raft_consensus.cc:2272] T 055432f1640d4d5a8a4ec2355e0b89ac P 438159d8de3c4027bdcf57eb81dedf86 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.325057 22629 tablet_replica.cc:333] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.325842 22629 raft_consensus.cc:2243] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:11.327167 22629 raft_consensus.cc:2272] T 9d72845b5ad4457bb71a7ad5f5152b4c P 438159d8de3c4027bdcf57eb81dedf86 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:11.337666 22629 tablet_replica.cc:333] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86: stopping tablet replica
I20260709 12:16:11.338337 22629 raft_consensus.cc:2243] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:11.339250 22629 raft_consensus.cc:2272] T c6f2d1f3ed104c1e99f82ebe04843248 P 438159d8de3c4027bdcf57eb81dedf86 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:11.392203 23899 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.513751 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.518181 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.599085 25114 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:11.630802 23899 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:11.724859 23899 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:12.273434 30680 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.274569 30680 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.353317 30680 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.332852 30681 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.398466 30681 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.401193 30681 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.403602 30646 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.404037 30656 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.404855 30656 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.407056 30656 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.404491 30646 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:12.428462 30646 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.477360 30661 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.478348 30661 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.483315 30677 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.484066 30677 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.485561 30688 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.486482 30677 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.481962 30661 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.437834 30656 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.439918 30643 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.488173 30656 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.444133 30686 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.489430 30686 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.486256 30688 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:16:12.488238 30643 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.442883 30691 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.436081 30684 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.491446 30656 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.501803 30683 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.502519 30683 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.492439 30648 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.506035 30648 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.492677 30686 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.493839 30682 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.492323 30690 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.494552 30688 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
I20260709 12:16:12.497138 30691 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:12.498634 30693 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.508448 30690 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.511688 30691 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.512526 30693 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:12.497710 30643 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.494201 30680 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.514428 30680 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.515839 30693 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.499632 30684 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.508455 30682 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.517401 30690 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.518812 30684 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.492003 30660 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.519852 30689 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.519981 30660 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.520685 30689 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.498922 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.522118 25175 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:16:12.523404 30689 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.525730 30689 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.526481 30689 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:12.528827 30689 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:12.530925 25114 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.534176 30680 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.532307 30682 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.525667 30648 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.536614 30660 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.533681 30683 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.541824 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.543774 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.544409 25175 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 13.
I20260709 12:16:12.546497 30689 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.547240 30689 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:12.549976 30689 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.551129 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.551846 25175 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:16:12.552457 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.553125 23932 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:16:12.554592 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.555138 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.555222 23932 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 15.
I20260709 12:16:12.556449 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.556617 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.557195 23931 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 13.
I20260709 12:16:12.557996 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.557365 23932 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:16:12.558425 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.559345 23925 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 19.
I20260709 12:16:12.554653 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.560601 25172 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 15.
I20260709 12:16:12.563920 25175 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 21.
W20260709 12:16:12.534466 25114 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:12.567024 23900 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 68419.8us
I20260709 12:16:12.567672 30680 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.568379 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.568810 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.568354 30680 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.569533 30682 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:12.569612 25172 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 19.
I20260709 12:16:12.569985 30682 raft_consensus.cc:493] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.570039 30683 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.570465 30682 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:12.570974 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.570842 30683 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.571791 25175 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
W20260709 12:16:12.575665 23900 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.580389 25113 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.581377 30660 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:12.581697 25113 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.581786 30660 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.582324 30660 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:12.582588 30648 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:12.583024 30648 raft_consensus.cc:493] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.583392 30648 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:12.590605 30648 raft_consensus.cc:515] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.582332 30682 raft_consensus.cc:515] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.593371 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.594270 30643 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:12.594657 30643 raft_consensus.cc:493] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.595014 30643 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:12.596469 30680 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.598560 30683 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:12.599889 30683 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.600472 30683 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.602772 30683 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.607678 30643 raft_consensus.cc:515] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:12.608031 25119 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.621333 25119 leader_election.cc:336] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.622116 25119 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.622799 25119 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.623457 25119 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.622926 30643 leader_election.cc:290] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:12.616233 23900 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.623986 23925 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
W20260709 12:16:12.623960 25119 leader_election.cc:336] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.624615 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:12.624895 25119 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:12.625118 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98e5bc86804f4f979f58779caa696814" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:12.625300 23929 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 10.
W20260709 12:16:12.625408 23899 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.625697 30683 raft_consensus.cc:2749] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:16:12.626084 23899 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.626125 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1b618b31b2e427b8c073b2c24771903" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:12.626375 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8b83386d2fa4ed2a212fbe8873239ea" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:16:12.626832 25119 leader_election.cc:336] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.627086 23931 raft_consensus.cc:3060] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:12.625702 23932 raft_consensus.cc:3060] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:16:12.627483 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:12.628580 23899 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.628942 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:12.629865 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.630319 30684 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:12.619720 30648 leader_election.cc:290] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.613382 30682 leader_election.cc:290] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:12.630339 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.624711 23925 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
W20260709 12:16:12.631700 23899 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.627188 23934 raft_consensus.cc:3060] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:12.633044 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.633908 30682 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:16:12.634272 30682 raft_consensus.cc:493] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.634615 30682 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:12.636281 23931 raft_consensus.cc:2468] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 12.
I20260709 12:16:12.642057 30682 raft_consensus.cc:515] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.644739 25113 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.645437 23932 raft_consensus.cc:2468] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 14.
I20260709 12:16:12.645634 30648 raft_consensus.cc:2804] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:16:12.591267 30660 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.646186 30648 raft_consensus.cc:697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:12.647297 30648 consensus_queue.cc:237] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.649266 23934 raft_consensus.cc:2468] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 20.
W20260709 12:16:12.650906 25119 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.663451 25119 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.664265 30643 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:12.664652 25119 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.665131 25119 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.665836 30643 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:12.652515 23898 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.652647 25113 leader_election.cc:304] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.668222 23898 leader_election.cc:304] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.668560 30643 raft_consensus.cc:2804] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:12.668946 30684 raft_consensus.cc:2764] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:16:12.669134 30643 raft_consensus.cc:697] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:12.671180 23898 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.661953 30660 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.672120 23898 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.672333 25113 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.672587 30690 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:12.672964 30690 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.673197 30660 raft_consensus.cc:2804] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:16:12.673544 23900 leader_election.cc:336] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.674253 30690 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:12.669713 30684 raft_consensus.cc:2764] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:16:12.670840 30643 consensus_queue.cc:237] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.678071 30682 leader_election.cc:290] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:12.674590 30660 raft_consensus.cc:697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:12.679665 30660 consensus_queue.cc:237] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.680088 23932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7203e0ca92840388d931c03974d32ff" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
I20260709 12:16:12.680780 23932 raft_consensus.cc:3060] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:12.681401 30693 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:12.681816 30693 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.682169 30693 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:12.687115 23679 catalog_manager.cc:5697] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 11 to 12, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 12 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:12.688380 23898 leader_election.cc:304] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
W20260709 12:16:12.689563 25119 leader_election.cc:336] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.691283 25119 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.694073 25119 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.694789 25119 leader_election.cc:336] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.695462 25119 leader_election.cc:336] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.696187 23934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da"
W20260709 12:16:12.696192 25119 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.696939 25119 leader_election.cc:336] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.697207 30684 raft_consensus.cc:2764] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:16:12.698607 30684 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.699118 30684 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.701339 30684 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.696904 30693 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.703155 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:12.703759 25175 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 11.
I20260709 12:16:12.704496 30693 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.705650 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.707036 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78f1c4dc751a4f4a93a446a785edda51" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:16:12.707548 25175 raft_consensus.cc:3060] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:12.712651 30693 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:12.713121 30693 raft_consensus.cc:493] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Starting leader election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:12.713493 30693 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:12.714001 25175 raft_consensus.cc:2468] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 22.
I20260709 12:16:12.715466 23898 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.716377 30684 raft_consensus.cc:2804] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:12.716962 30684 raft_consensus.cc:697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 22 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
W20260709 12:16:12.722635 23900 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.723148 23900 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.723918 30689 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:12.723549 30690 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.724934 23934 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 05679ef5e2b74861aad2a08b6a05e276 in current term 16: Already voted for candidate 9e0bf56d58c5480a8d088c85566558da in this term.
W20260709 12:16:12.725127 23900 leader_election.cc:336] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.725548 23900 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:12.726881 23900 leader_election.cc:336] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.727792 30684 consensus_queue.cc:237] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.730822 30690 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.731308 25113 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:12.731823 25114 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.732970 25175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
W20260709 12:16:12.738354 23900 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.739562 30693 raft_consensus.cc:515] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.752832 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9f2f570298740c691fa19d449b3fa4f" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:16:12.753480 25172 raft_consensus.cc:3060] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:12.741848 23932 raft_consensus.cc:2468] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 11.
I20260709 12:16:12.747742 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:12.761271 23900 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.762071 23900 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.762240 30693 leader_election.cc:290] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:12.747133 30643 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:16:12.761991 25172 raft_consensus.cc:2468] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 12.
I20260709 12:16:12.763608 30689 raft_consensus.cc:2749] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:16:12.764256 23898 leader_election.cc:304] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:12.765129 30690 raft_consensus.cc:2804] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:16:12.765862 30690 raft_consensus.cc:697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:12.766925 30690 consensus_queue.cc:237] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.769394 25175 raft_consensus.cc:2393] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e0bf56d58c5480a8d088c85566558da in current term 16: Already voted for candidate 05679ef5e2b74861aad2a08b6a05e276 in this term.
I20260709 12:16:12.747330 25114 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.762938 30693 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:12.772773 30660 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:12.774642 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.775198 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.776034 30690 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:12.762540 23900 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.776938 30690 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:12.767632 25113 leader_election.cc:304] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.782238 30660 raft_consensus.cc:2804] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:16:12.782780 30660 raft_consensus.cc:697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Becoming Leader. State: Replica: 05679ef5e2b74861aad2a08b6a05e276, State: Running, Role: LEADER
I20260709 12:16:12.783725 30660 consensus_queue.cc:237] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:12.786808 23676 catalog_manager.cc:5697] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 21 to 22, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 22 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:12.790062 23900 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.783738 25114 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:12.790845 25114 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.791904 30643 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:12.790817 23900 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 05679ef5e2b74861aad2a08b6a05e276, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.793810 30693 raft_consensus.cc:2749] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:16:12.794045 23900 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.793094 25114 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:12.794947 23900 leader_election.cc:336] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.794955 25114 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.795969 30643 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:12.795994 23900 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.796708 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.797499 30693 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:12.798010 23900 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.798480 23900 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:12.799099 30693 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:12.799285 23900 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.800062 23900 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.800364 23900 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:12.801293 23900 leader_election.cc:336] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:12.801728 30693 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:12.804610 23900 leader_election.cc:336] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:12.952275 25119 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.604841 25119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.66:44897, user_credentials={real_user=slave}} blocked reactor thread for 652573us
I20260709 12:16:13.549862 23676 catalog_manager.cc:5697] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 13 to 14, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 14 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:13.553643 23679 catalog_manager.cc:5697] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 11 to 12, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 12 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:13.594846 25168 raft_consensus.cc:1275] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:16:13.601506 25170 raft_consensus.cc:1275] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 12:16:13.501427 23714 debug-util.cc:398] Leaking SignalData structure 0x7b08001769a0 after lost signal to thread 22632
W20260709 12:16:13.615887 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080039bde0 after lost signal to thread 23717
W20260709 12:16:13.617280 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080003a9a0 after lost signal to thread 23958
W20260709 12:16:13.558560 23899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111) [suppressed 475 similar messages]
W20260709 12:16:13.521553 22629 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 23881)
W20260709 12:16:13.559978 25114 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:13.619369 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 59451.3us
W20260709 12:16:13.619935 25114 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:13.622129 30713 consensus_queue.cc:1048] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:13.606606 23676 catalog_manager.cc:5697] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 19 to 20, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 20 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:13.625531 30715 consensus_queue.cc:1048] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:13.624493 23676 catalog_manager.cc:5697] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 reported cstate change: term changed from 10 to 11, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68). New cstate: current_term: 11 leader_uuid: "05679ef5e2b74861aad2a08b6a05e276" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:13.624377 23714 debug-util.cc:398] Leaking SignalData structure 0x7b080003abe0 after lost signal to thread 25207
W20260709 12:16:13.677853 23900 consensus_peers.cc:597] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.680159 23899 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 121625us
W20260709 12:16:13.684665 23900 consensus_peers.cc:597] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.685842 23900 consensus_peers.cc:597] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.702947 23931 raft_consensus.cc:1275] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:16:13.711412 23900 consensus_peers.cc:597] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.722684 30712 consensus_queue.cc:1048] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:13.724470 23931 raft_consensus.cc:1275] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:13.764546 30716 consensus_queue.cc:1048] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
W20260709 12:16:13.835090 25114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.65:35817, user_credentials={real_user=slave}} blocked reactor thread for 115101us
W20260709 12:16:13.844537 25119 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.848595 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 117687us
I20260709 12:16:13.869750 23931 raft_consensus.cc:1275] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:16:13.872259 23929 raft_consensus.cc:1275] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Refusing update from remote peer 05679ef5e2b74861aad2a08b6a05e276: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:16:13.873479 30717 consensus_queue.cc:1048] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:16:13.874522 25119 consensus_peers.cc:597] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:13.879844 30737 consensus_queue.cc:1048] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:16:13.886938 25114 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:13.903774 25114 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:13.908460 23679 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58558 (request call id 408) took 1082 ms. Trace:
W20260709 12:16:13.909633 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:13.909030 23679 rpcz_store.cc:276] 0709 12:16:12.825835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:12.825988 (+   153us) service_pool.cc:224] Handling call
0709 12:16:12.829020 (+  3032us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:16:13.908391 (+1079371us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:13.728573 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:13.729132 (+   559us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:13.729162 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:16:13.729181 (+    19us) tablet.cc:662] Decoding operations
0709 12:16:13.729494 (+   313us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:13.729621 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:13.729647 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:13.729831 (+   184us) tablet.cc:701] Row locks acquired
0709 12:16:13.729847 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:16:13.734102 (+  4255us) write_op.cc:270] Start()
0709 12:16:13.734232 (+   130us) write_op.cc:276] Timestamp: P: 1783599373734036 usec, L: 0
0709 12:16:13.734256 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:16:13.735003 (+   747us) log.cc:844] Serialized 533 byte log entry
0709 12:16:13.845164 (+110161us) op_driver.cc:464] REPLICATION: finished
0709 12:16:13.869493 (+ 24329us) write_op.cc:301] APPLY: starting
0709 12:16:13.869589 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:13.894165 (+ 24576us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:13.894640 (+   475us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:13.895374 (+   734us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:13.895563 (+   189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:16:13.895625 (+    62us) write_op.cc:312] APPLY: finished
0709 12:16:13.895822 (+   197us) log.cc:844] Serialized 25 byte log entry
0709 12:16:13.896004 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:13.896079 (+    75us) write_op.cc:454] Released schema lock
0709 12:16:13.896117 (+    38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":4141952,"child_traces":[["op",{"apply.queue_time_us":24047,"apply.run_cpu_time_us":3665,"apply.run_wall_time_us":40428,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":117809,"prepare.run_cpu_time_us":2578,"prepare.run_wall_time_us":6715,"replication_time_us":109959,"thread_start_us":141604,"threads_started":2}]]}
W20260709 12:16:13.936556 25119 consensus_peers.cc:597] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.946426 23900 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.953344 23899 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:13.959851 25119 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.961084 23676 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50498 (request call id 318) took 1140 ms. Trace:
I20260709 12:16:13.974951 23676 rpcz_store.cc:276] 0709 12:16:12.820508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:12.820661 (+   153us) service_pool.cc:224] Handling call
0709 12:16:12.828907 (+  8246us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b5400038254. stack: 00007fb2aa693343 00007fb2aa693240 00007fb2aa0221d4 00007fb2b3d60304 00007fb2b3d60299 00007fb2ad71c090 00007fb2ad7baf3a 00007fb2b3570e9c 00007fb2b366f04e 00007fb2b1b302a4 00007fb2b1b30232 00007fb2b1b30161 00007fb2b1b300dc 00007fb2b1b2f392 00007fb2abaed48c 00007fb2abaec8c6
0709 12:16:13.961010 (+1132103us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:13.735467 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:13.735990 (+   523us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:13.736033 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:16:13.736045 (+    12us) tablet.cc:662] Decoding operations
0709 12:16:13.736553 (+   508us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:13.736607 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:13.736633 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:16:13.736940 (+   307us) tablet.cc:701] Row locks acquired
0709 12:16:13.736952 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:16:13.737101 (+   149us) write_op.cc:270] Start()
0709 12:16:13.737199 (+    98us) write_op.cc:276] Timestamp: P: 1783599373737054 usec, L: 0
0709 12:16:13.737214 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:16:13.737773 (+   559us) log.cc:844] Serialized 1293 byte log entry
0709 12:16:13.877982 (+140209us) op_driver.cc:464] REPLICATION: finished
0709 12:16:13.910304 (+ 32322us) write_op.cc:301] APPLY: starting
0709 12:16:13.910377 (+    73us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:13.924925 (+ 14548us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:13.925024 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:13.941861 (+ 16837us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:13.942058 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:16:13.942106 (+    48us) write_op.cc:312] APPLY: finished
0709 12:16:13.942334 (+   228us) log.cc:844] Serialized 41 byte log entry
0709 12:16:13.958610 (+ 16276us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:13.958711 (+   101us) write_op.cc:454] Released schema lock
0709 12:16:13.958759 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:13.959131 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:13.959159 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":15588608,"child_traces":[["op",{"apply.queue_time_us":31893,"apply.run_cpu_time_us":5992,"apply.run_wall_time_us":48792,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12546,"num_ops":3,"prepare.queue_time_us":104920,"prepare.run_cpu_time_us":2564,"prepare.run_wall_time_us":2564,"replication_time_us":134535,"thread_start_us":53671,"threads_started":1}]]}
W20260709 12:16:13.971546 23900 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.974978 25119 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.984274 25119 consensus_peers.cc:597] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.985055 25119 consensus_peers.cc:597] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.985693 25119 consensus_peers.cc:597] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.986243 25119 consensus_peers.cc:597] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.995285 23900 consensus_peers.cc:597] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.997987 23900 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.003481 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.012178 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.015794 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.021802 23900 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.049037 23900 consensus_peers.cc:597] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.064177 23900 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.068477 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.081631 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.084712 25114 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:14.111629 25114 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:14.112155 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.114645 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.117276 23900 consensus_peers.cc:597] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.122314 23899 consensus_peers.cc:597] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.122740 25119 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.123124 23900 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.146519 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.190928 23900 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.227538 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.235453 25119 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.320397 30706 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.320996 30706 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
W20260709 12:16:14.328078 23900 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.329322 30706 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:16:14.333999 23899 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.334441 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:14.336549 30715 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:14.394778 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.415459 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.415634 25119 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.465602 30737 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.466495 30737 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:14.470093 23899 consensus_peers.cc:597] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:14.492656 23929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "861b3b9e09c14ef99794d4c8d50d59ad" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:14.493432 23929 raft_consensus.cc:2468] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 17.
I20260709 12:16:14.495245 30706 raft_consensus.cc:493] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.495865 30706 raft_consensus.cc:515] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:14.498056 30706 leader_election.cc:290] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:14.508075 30712 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.508671 30712 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:14.510602 23900 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:14.511289 23900 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.512176 30712 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:14.515230 25119 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.520988 30737 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:14.529975 23899 consensus_peers.cc:597] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:14.533097 23931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c395cd7fe30b41869bce6dac359f2cf8" candidate_uuid: "05679ef5e2b74861aad2a08b6a05e276" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9e0bf56d58c5480a8d088c85566558da" is_pre_election: true
I20260709 12:16:14.533739 23931 raft_consensus.cc:2468] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05679ef5e2b74861aad2a08b6a05e276 in term 16.
W20260709 12:16:14.534906 25119 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:14.537031 25114 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.541805 25114 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:14.545946 23899 leader_election.cc:336] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.546802 23899 leader_election.cc:304] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:14.547363 30737 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:14.547541 30706 raft_consensus.cc:2749] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:14.548403 25119 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.549100 25114 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:14.550648 25119 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.551126 25119 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:14.553413 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800399260 after lost signal to thread 22632
W20260709 12:16:14.566177 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080031ddc0 after lost signal to thread 23717
W20260709 12:16:14.567548 25204 debug-util.cc:398] Leaking SignalData structure 0x7b0800045380 after lost signal to thread 23958
W20260709 12:16:14.568781 25204 debug-util.cc:398] Leaking SignalData structure 0x7b080015d060 after lost signal to thread 25207
W20260709 12:16:14.556566 23899 consensus_peers.cc:597] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:14.578910 30737 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:14.582769 30706 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.583384 30706 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:14.585516 30706 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.591873 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:14.593947 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.619021 22629 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 23881)
W20260709 12:16:14.661592 25119 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.686700 25119 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:14.692118 23900 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.696378 23900 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:14.700121 30721 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:14.696973 23899 consensus_peers.cc:597] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:14.706523 30721 raft_consensus.cc:493] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.707696 30721 raft_consensus.cc:515] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:14.711875 30721 leader_election.cc:290] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.717784 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:14.732611 30721 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.735886 30737 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.739228 30737 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:14.742446 30737 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
I20260709 12:16:14.743165 30721 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:14.748932 30715 raft_consensus.cc:493] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.756551 30715 raft_consensus.cc:515] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:14.759044 30715 leader_election.cc:290] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.750697 23899 consensus_peers.cc:597] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:14.767455 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276" is_pre_election: true
I20260709 12:16:14.768345 25172 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 16.
I20260709 12:16:14.769985 23898 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:14.770967 30715 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:14.771402 30715 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.771855 30715 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:14.787519 23900 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.798835 30721 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:14.800585 30737 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.811323 30737 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:14.814738 30737 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.816025 25119 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.807377 30715 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:14.835567 23899 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.839217 23899 leader_election.cc:336] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.837980 25172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053977846bfa4adf9703ea669a81f41d" candidate_uuid: "9e0bf56d58c5480a8d088c85566558da" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "05679ef5e2b74861aad2a08b6a05e276"
I20260709 12:16:14.841751 25172 raft_consensus.cc:3060] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:14.849957 25172 raft_consensus.cc:2468] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e0bf56d58c5480a8d088c85566558da in term 17.
I20260709 12:16:14.854684 23898 leader_election.cc:304] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276, 9e0bf56d58c5480a8d088c85566558da; no voters: 
I20260709 12:16:14.836907 30715 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: Requested vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 05679ef5e2b74861aad2a08b6a05e276 (127.22.25.68:43683)
I20260709 12:16:14.855954 30715 raft_consensus.cc:2804] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:14.857561 30715 raft_consensus.cc:697] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Becoming Leader. State: Replica: 9e0bf56d58c5480a8d088c85566558da, State: Running, Role: LEADER
I20260709 12:16:14.858650 30715 consensus_queue.cc:237] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:14.863502 23899 leader_election.cc:304] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:14.863538 25119 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.865154 30721 raft_consensus.cc:2749] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:14.871057 23900 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:14.872830 25114 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:14.875311 23900 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:14.878263 25114 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:14.882102 30737 raft_consensus.cc:2749] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:14.891837 23900 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:14.900913 23899 leader_election.cc:336] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.901490 23899 leader_election.cc:304] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:14.902475 23899 consensus_peers.cc:597] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:14.902909 23899 consensus_peers.cc:597] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:14.903496 30706 raft_consensus.cc:2749] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:14.921483 30721 raft_consensus.cc:493] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.922227 30721 raft_consensus.cc:515] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
I20260709 12:16:14.926172 30721 leader_election.cc:290] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.936075 23899 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:14.940657 30712 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:14.941340 30712 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:14.943882 30712 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:14.974025 25119 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:15.002442 25114 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:15.003006 25114 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:15.003381 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:15.004894 23679 catalog_manager.cc:5697] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da reported cstate change: term changed from 15 to 17, leader changed from 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66) to 9e0bf56d58c5480a8d088c85566558da (127.22.25.67). New cstate: current_term: 17 leader_uuid: "9e0bf56d58c5480a8d088c85566558da" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:15.009163 25114 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:15.009614 25114 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
W20260709 12:16:15.012837 23900 leader_election.cc:336] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:15.014119 23900 leader_election.cc:304] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:15.015947 30751 raft_consensus.cc:2749] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:15.019730 30752 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:15.012290 30714 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:15.191236 25114 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:15.258787 23899 consensus_peers.cc:597] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:15.273654 23899 consensus_peers.cc:597] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:15.322279 23900 consensus_peers.cc:597] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:15.418745 25170 raft_consensus.cc:1275] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Refusing update from remote peer 9e0bf56d58c5480a8d088c85566558da: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:15.421763 30751 consensus_queue.cc:1048] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [LEADER]: Connected to new peer: Peer: permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:16:15.805953 23900 consensus_peers.cc:597] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:15.820883 25119 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:15.962544 25119 consensus_peers.cc:597] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:16.024492 25114 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:16.042657 23900 consensus_peers.cc:597] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.053049 22629 tablet_server.cc:196] TabletServer@127.22.25.66:0 shutdown complete.
I20260709 12:16:16.081116 30751 raft_consensus.cc:493] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:16.081840 30751 raft_consensus.cc:515] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } }
I20260709 12:16:16.085685 30751 leader_election.cc:290] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817)
W20260709 12:16:16.098771 23900 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:16.102535 23899 leader_election.cc:336] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:16.103143 23899 leader_election.cc:304] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:16.104070 30751 raft_consensus.cc:2749] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:16.148401 25119 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.155195 25114 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:16.224237 25119 consensus_peers.cc:597] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:16.269863 22629 tablet_server.cc:179] TabletServer@127.22.25.67:0 shutting down...
W20260709 12:16:16.272159 25113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.22.25.67:40463, user_credentials={real_user=slave}} blocked reactor thread for 98884us
W20260709 12:16:16.291078 25113 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.293397 25119 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.295300 25119 consensus_peers.cc:597] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:16.304906 25114 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:16.310573 25114 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:16.336812 30751 raft_consensus.cc:493] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:16.337410 30751 raft_consensus.cc:515] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } }
W20260709 12:16:16.345472 25119 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.348405 30751 leader_election.cc:290] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:16.349325 23900 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:16.360682 23899 consensus_peers.cc:597] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.361199 23899 leader_election.cc:336] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:16.361609 23899 leader_election.cc:304] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e0bf56d58c5480a8d088c85566558da; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:16.362301 30751 raft_consensus.cc:2749] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:16.366192 25113 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.370656 23900 consensus_peers.cc:597] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:16.373442 23900 consensus_peers.cc:597] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.382198 25113 consensus_peers.cc:597] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.386597 23900 consensus_peers.cc:597] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.388276 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:16.389355 22629 tablet_replica.cc:333] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.390316 22629 raft_consensus.cc:2243] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.391067 22629 raft_consensus.cc:2272] T 349b687db5c44196bafc207c26ca3679 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.394794 22629 tablet_replica.cc:333] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.395749 22629 raft_consensus.cc:2243] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.397357 22629 raft_consensus.cc:2272] T 053977846bfa4adf9703ea669a81f41d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.401057 25113 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.401309 23900 consensus_peers.cc:597] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.407454 25113 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.408470 23900 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.410724 25113 consensus_peers.cc:597] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.410980 22629 tablet_replica.cc:333] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.412143 22629 raft_consensus.cc:2243] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.412894 22629 raft_consensus.cc:2272] T b7caafee6779404494f8df62d888c5ef P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.415813 25113 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.415982 22629 tablet_replica.cc:333] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.416860 22629 raft_consensus.cc:2243] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.417569 22629 raft_consensus.cc:2272] T e2366a1e0cb342ee891c776affbfbca3 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.420348 22629 tablet_replica.cc:333] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.421120 22629 raft_consensus.cc:2243] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.421710 22629 raft_consensus.cc:2272] T e76b9730b35144689e48d4d145fabb18 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.424835 22629 tablet_replica.cc:333] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.425681 22629 raft_consensus.cc:2243] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.426329 22629 raft_consensus.cc:2272] T 956a6729b1f34c499670aa77b5896a53 P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.430508 22629 tablet_replica.cc:333] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.431336 22629 raft_consensus.cc:2243] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.433115 22629 raft_consensus.cc:2272] T a1b618b31b2e427b8c073b2c24771903 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.435674 22629 tablet_replica.cc:333] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
W20260709 12:16:16.436129 25114 consensus_peers.cc:597] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:16.436482 22629 raft_consensus.cc:2243] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.437747 22629 raft_consensus.cc:2272] T 1567865ea02a4884996ae9d8beb9fa29 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.441054 22629 tablet_replica.cc:333] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.441887 22629 raft_consensus.cc:2243] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 20 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.442770 23899 consensus_peers.cc:597] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:16.443176 22629 raft_consensus.cc:2272] T 34e5e0c10245477d9010ece72d08ae51 P 9e0bf56d58c5480a8d088c85566558da [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.445899 22629 tablet_replica.cc:333] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.446810 22629 raft_consensus.cc:2243] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.448028 22629 raft_consensus.cc:2272] T e943fd8cd04e4a5e9678a4bdbd7b1449 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.451133 22629 tablet_replica.cc:333] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.452086 22629 raft_consensus.cc:2243] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.454406 22629 raft_consensus.cc:2272] T d8341d6ef4764792b1bd6ff196e8af38 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.459914 22629 tablet_replica.cc:333] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.460743 22629 raft_consensus.cc:2243] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:16.461150 25114 consensus_peers.cc:597] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:16.461330 22629 raft_consensus.cc:2272] T 233089f4b82a4b1996ad9129dc392819 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.464272 22629 tablet_replica.cc:333] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.465080 22629 raft_consensus.cc:2243] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.465664 22629 raft_consensus.cc:2272] T c8b83386d2fa4ed2a212fbe8873239ea P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.468053 25113 consensus_peers.cc:597] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.468248 22629 tablet_replica.cc:333] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.468956 22629 raft_consensus.cc:2243] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.469849 22629 raft_consensus.cc:2272] T f5bb29269e8a4aa1b79f91e4ed687053 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.472002 22629 tablet_replica.cc:333] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.472610 22629 raft_consensus.cc:2243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.473170 22629 raft_consensus.cc:2272] T 8f44bb7910714c46af4e3d0e2076e7c5 P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.475476 22629 tablet_replica.cc:333] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.476301 22629 raft_consensus.cc:2243] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.477619 22629 raft_consensus.cc:2272] T 1b93ab9c24064567b717af205bc2ed2a P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.482378 22629 tablet_replica.cc:333] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.483273 22629 raft_consensus.cc:2243] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.488545 22629 raft_consensus.cc:2272] T 3eeb67716b0e4f49aa1a4bd5815f684b P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.491801 22629 tablet_replica.cc:333] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.492619 22629 raft_consensus.cc:2243] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.493420 23900 consensus_peers.cc:597] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.493796 22629 raft_consensus.cc:2272] T 20edc48b2f884789ac3daead4b5c3102 P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.496826 22629 tablet_replica.cc:333] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.497601 22629 raft_consensus.cc:2243] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.499724 25113 consensus_peers.cc:597] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.500819 22629 raft_consensus.cc:2272] T 526cc198da134bb590358ed73c23d59e P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.504101 22629 tablet_replica.cc:333] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.504940 22629 raft_consensus.cc:2243] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.506119 22629 raft_consensus.cc:2272] T e1031da3c683416ebfc283d98b45867d P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.507651 25113 consensus_peers.cc:597] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.509068 22629 tablet_replica.cc:333] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.509907 22629 raft_consensus.cc:2243] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.510885 22629 raft_consensus.cc:2272] T c2844521776744308787188de9041ebd P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.513325 22629 tablet_replica.cc:333] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.513950 22629 raft_consensus.cc:2243] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.514860 22629 raft_consensus.cc:2272] T c395cd7fe30b41869bce6dac359f2cf8 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.517076 22629 tablet_replica.cc:333] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.517890 22629 raft_consensus.cc:2243] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.519096 22629 raft_consensus.cc:2272] T 8368f2e695204683affed6a9bd84ba96 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.523801 25119 consensus_peers.cc:597] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.524180 22629 tablet_replica.cc:333] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.526382 22629 raft_consensus.cc:2243] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.526934 22629 raft_consensus.cc:2272] T c7203e0ca92840388d931c03974d32ff P 9e0bf56d58c5480a8d088c85566558da [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.529047 22629 tablet_replica.cc:333] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.529843 22629 raft_consensus.cc:2243] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.530506 22629 raft_consensus.cc:2272] T 7f5f207bbfed4f119db8464be0a5a8cf P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.533257 22629 tablet_replica.cc:333] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.534401 22629 raft_consensus.cc:2243] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.535487 22629 raft_consensus.cc:2272] T d9f2f570298740c691fa19d449b3fa4f P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.540577 23899 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:16.542622 22629 tablet_replica.cc:333] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.543614 22629 raft_consensus.cc:2243] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.544764 22629 raft_consensus.cc:2272] T cc9ef29919aa42aaa2b7545c10e39eff P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.547550 22629 tablet_replica.cc:333] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.548339 22629 raft_consensus.cc:2243] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.549714 22629 raft_consensus.cc:2272] T ed3c66ee31004577b517e2c4322696b6 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.551822 23900 consensus_peers.cc:597] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:16.552543 22629 tablet_replica.cc:333] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.553474 22629 raft_consensus.cc:2243] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.553596 25113 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.554500 22629 raft_consensus.cc:2272] T 10c8a875cf4b4acea62c1086994499e4 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.557250 22629 tablet_replica.cc:333] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.558152 22629 raft_consensus.cc:2243] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.559391 22629 raft_consensus.cc:2272] T a979dd022fc24afd8765a3cdb5f7fab3 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.560737 25113 consensus_peers.cc:597] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.561880 22629 tablet_replica.cc:333] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.562502 22629 raft_consensus.cc:2243] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.563539 22629 raft_consensus.cc:2272] T 77aa2cbe928a45d4b1652ca10a80aa99 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.565953 22629 tablet_replica.cc:333] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.566634 22629 raft_consensus.cc:2243] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.567466 22629 raft_consensus.cc:2272] T bec362a6d01840d39be46f0c3d28c218 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.569846 22629 tablet_replica.cc:333] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.570420 22629 raft_consensus.cc:2243] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.571250 22629 raft_consensus.cc:2272] T de3c70defd864769b26b52b7f36548cd P 9e0bf56d58c5480a8d088c85566558da [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.573362 22629 tablet_replica.cc:333] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.573927 22629 raft_consensus.cc:2243] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.574424 22629 raft_consensus.cc:2272] T 872245a7005f472bb7f6cfafda680b74 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.576754 22629 tablet_replica.cc:333] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.577291 22629 raft_consensus.cc:2243] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.578224 22629 raft_consensus.cc:2272] T 98e5bc86804f4f979f58779caa696814 P 9e0bf56d58c5480a8d088c85566558da [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.580267 22629 tablet_replica.cc:333] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.580945 22629 raft_consensus.cc:2243] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.581450 22629 raft_consensus.cc:2272] T b54dd3a3d61243bea2dae318ccea3958 P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.583518 22629 tablet_replica.cc:333] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.584095 22629 raft_consensus.cc:2243] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.584995 22629 raft_consensus.cc:2272] T 0c9d6097cb8d40efa55ff97de5b26130 P 9e0bf56d58c5480a8d088c85566558da [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.587062 22629 tablet_replica.cc:333] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.587584 22629 raft_consensus.cc:2243] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.588186 22629 raft_consensus.cc:2272] T 3aacc8d80dd2433b94e6d07927304b47 P 9e0bf56d58c5480a8d088c85566558da [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.590173 22629 tablet_replica.cc:333] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.590705 22629 raft_consensus.cc:2243] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:16.591921 22629 raft_consensus.cc:2272] T 78f1c4dc751a4f4a93a446a785edda51 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.593845 22629 tablet_replica.cc:333] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.594376 22629 raft_consensus.cc:2243] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.594956 22629 raft_consensus.cc:2272] T 0b23f5ca4f084b8e96cc4a3099e99074 P 9e0bf56d58c5480a8d088c85566558da [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.596874 22629 tablet_replica.cc:333] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.597371 22629 raft_consensus.cc:2243] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.597838 22629 raft_consensus.cc:2272] T 9d72845b5ad4457bb71a7ad5f5152b4c P 9e0bf56d58c5480a8d088c85566558da [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.599849 22629 tablet_replica.cc:333] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.600315 22629 raft_consensus.cc:2243] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.600804 22629 raft_consensus.cc:2272] T c6f2d1f3ed104c1e99f82ebe04843248 P 9e0bf56d58c5480a8d088c85566558da [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.602837 22629 tablet_replica.cc:333] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.603371 22629 raft_consensus.cc:2243] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.604236 22629 raft_consensus.cc:2272] T 861b3b9e09c14ef99794d4c8d50d59ad P 9e0bf56d58c5480a8d088c85566558da [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.606530 22629 tablet_replica.cc:333] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.607149 22629 raft_consensus.cc:2243] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.607770 22629 raft_consensus.cc:2272] T ef461146a2924dba8b8e2906d76859f1 P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:16.610548 22629 tablet_replica.cc:333] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da: stopping tablet replica
I20260709 12:16:16.611155 22629 raft_consensus.cc:2243] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 LEADER]: Raft consensus shutting down.
W20260709 12:16:16.612134 25114 consensus_peers.cc:597] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:16.612346 22629 raft_consensus.cc:2272] T 5ed3fc49956b40aa86095410b7c7e5fa P 9e0bf56d58c5480a8d088c85566558da [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.623353 25113 consensus_peers.cc:597] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.632582 25119 consensus_peers.cc:597] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:16.640413 25113 consensus_peers.cc:597] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:16.641093 25114 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.641602 25114 consensus_peers.cc:597] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.665613 25113 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Couldn't send request to peer 9e0bf56d58c5480a8d088c85566558da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:16.689663 30763 raft_consensus.cc:493] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:16.690246 30763 raft_consensus.cc:515] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:16.692652 30763 leader_election.cc:290] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:16.708063 25119 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:16.708886 25114 leader_election.cc:336] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:16.709335 25114 leader_election.cc:304] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:16.710549 30763 raft_consensus.cc:2749] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:16.729200 25114 consensus_peers.cc:597] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.732535 25114 consensus_peers.cc:597] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.763399 25119 consensus_peers.cc:597] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.777225 25119 consensus_peers.cc:597] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.799038 25114 consensus_peers.cc:597] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.869019 25119 consensus_peers.cc:597] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.887187 25114 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:16.943898 25119 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:16.974601 25114 consensus_peers.cc:597] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:16.981191 25119 consensus_peers.cc:597] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:17.017745 25114 consensus_peers.cc:597] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:17.044602 25114 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:17.104539 30766 raft_consensus.cc:493] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:17.105170 30766 raft_consensus.cc:515] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:17.114789 25119 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:17.116905 30766 leader_election.cc:290] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.125723 25113 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.126219 25113 leader_election.cc:304] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:17.127177 30766 raft_consensus.cc:2749] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:17.128139 25114 leader_election.cc:336] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:17.137456 25119 consensus_peers.cc:597] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:17.158474 30766 raft_consensus.cc:493] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:17.183854 30766 raft_consensus.cc:515] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } attrs { replace: true } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.211233 30766 leader_election.cc:290] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.230504 25119 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
W20260709 12:16:17.236153 25114 consensus_peers.cc:597] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:17.237102 25113 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.237565 25113 leader_election.cc:304] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:17.246975 30766 raft_consensus.cc:2749] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:17.248193 25114 leader_election.cc:336] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:17.264165 25119 consensus_peers.cc:597] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:17.286123 25114 consensus_peers.cc:597] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:17.409953 25119 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Couldn't send request to peer 438159d8de3c4027bdcf57eb81dedf86. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:17.411613 25114 consensus_peers.cc:597] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:17.449172 30764 raft_consensus.cc:493] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:17.449733 30764 raft_consensus.cc:515] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.452083 30764 leader_election.cc:290] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:17.470618 25114 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:17.471423 25119 leader_election.cc:336] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Network error: Client connection negotiation failed: client connection to 127.22.25.66:44897: connect: Connection refused (error 111)
I20260709 12:16:17.471910 25119 leader_election.cc:304] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:17.472683 30766 raft_consensus.cc:2749] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:17.550529 25114 consensus_peers.cc:597] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 -> Peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Couldn't send request to peer 0ee4ab0747914e7abc642312517b5e25. Status: Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:17.551364 30766 raft_consensus.cc:493] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.552004 30766 raft_consensus.cc:515] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.554178 30766 leader_election.cc:290] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.561229 25113 leader_election.cc:336] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.604949 22629 tablet_server.cc:196] TabletServer@127.22.25.67:0 shutdown complete.
I20260709 12:16:17.630671 30766 raft_consensus.cc:493] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.631435 30766 raft_consensus.cc:515] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.632416 30764 raft_consensus.cc:493] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.633039 30764 raft_consensus.cc:515] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.635413 30764 leader_election.cc:290] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:17.637372 30766 leader_election.cc:290] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.645371 25114 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:17.645998 25114 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
W20260709 12:16:17.646096 25113 leader_election.cc:336] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.646023 30766 raft_consensus.cc:493] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.646843 25113 leader_election.cc:304] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:17.647387 30766 raft_consensus.cc:515] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.647890 30764 raft_consensus.cc:2749] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:17.649093 25113 leader_election.cc:336] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.649583 25113 leader_election.cc:304] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:17.650244 30766 leader_election.cc:290] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:17.650384 30764 raft_consensus.cc:2749] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:17.656105 25113 leader_election.cc:336] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.665638 30764 raft_consensus.cc:493] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.666275 30764 raft_consensus.cc:515] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.668674 30764 leader_election.cc:290] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.673971 25113 leader_election.cc:336] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.681315 30764 raft_consensus.cc:493] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.681910 30764 raft_consensus.cc:515] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.684643 30764 leader_election.cc:290] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:17.689565 30764 raft_consensus.cc:493] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
W20260709 12:16:17.690003 25113 leader_election.cc:336] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.690225 30764 raft_consensus.cc:515] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.693995 30764 leader_election.cc:290] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
I20260709 12:16:17.697402 30764 raft_consensus.cc:493] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.698010 30764 raft_consensus.cc:515] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.700614 30764 leader_election.cc:290] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.703841 25113 leader_election.cc:336] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.704838 30764 raft_consensus.cc:493] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 438159d8de3c4027bdcf57eb81dedf86)
I20260709 12:16:17.705556 30764 raft_consensus.cc:515] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "438159d8de3c4027bdcf57eb81dedf86" member_type: VOTER last_known_addr { host: "127.22.25.66" port: 44897 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
W20260709 12:16:17.707046 25113 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
I20260709 12:16:17.709539 30764 leader_election.cc:290] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897)
W20260709 12:16:17.712397 25114 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:17.736297 22629 tablet_server.cc:179] TabletServer@127.22.25.68:0 shutting down...
I20260709 12:16:17.755710 30764 raft_consensus.cc:493] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9e0bf56d58c5480a8d088c85566558da)
I20260709 12:16:17.756455 30764 raft_consensus.cc:515] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "0ee4ab0747914e7abc642312517b5e25" member_type: VOTER last_known_addr { host: "127.22.25.65" port: 35817 } } peers { permanent_uuid: "9e0bf56d58c5480a8d088c85566558da" member_type: VOTER last_known_addr { host: "127.22.25.67" port: 40463 } } peers { permanent_uuid: "05679ef5e2b74861aad2a08b6a05e276" member_type: VOTER last_known_addr { host: "127.22.25.68" port: 43683 } attrs { promote: false } }
I20260709 12:16:17.762149 30764 leader_election.cc:290] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817), 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463)
W20260709 12:16:17.768095 25113 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9e0bf56d58c5480a8d088c85566558da (127.22.25.67:40463): Network error: Client connection negotiation failed: client connection to 127.22.25.67:40463: connect: Connection refused (error 111)
W20260709 12:16:17.769292 25114 leader_election.cc:336] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0ee4ab0747914e7abc642312517b5e25 (127.22.25.65:35817): Network error: Client connection negotiation failed: client connection to 127.22.25.65:35817: connect: Connection refused (error 111)
I20260709 12:16:17.769729 25114 leader_election.cc:304] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:17.770455 30764 raft_consensus.cc:2749] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:16:17.785244 22629 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:17.786062 22629 tablet_replica.cc:333] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.786713 22629 raft_consensus.cc:2243] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.787302 22629 raft_consensus.cc:2272] T 861b3b9e09c14ef99794d4c8d50d59ad P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.789685 22629 tablet_replica.cc:333] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.790374 22629 raft_consensus.cc:2243] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.791448 22629 raft_consensus.cc:2272] T fc09548ba8eb4f01bf6d74a33d072dda P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.793903 22629 tablet_replica.cc:333] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.794502 22629 raft_consensus.cc:2243] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.795085 22629 raft_consensus.cc:2272] T 053977846bfa4adf9703ea669a81f41d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.797542 22629 tablet_replica.cc:333] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.798136 22629 raft_consensus.cc:2243] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.798787 22629 raft_consensus.cc:2272] T 5038ac294ceb4758b5b5de70f2469d5d P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.801083 22629 tablet_replica.cc:333] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.801728 22629 raft_consensus.cc:2243] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.802685 22629 raft_consensus.cc:2272] T 1b93ab9c24064567b717af205bc2ed2a P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.805052 22629 tablet_replica.cc:333] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.805634 22629 raft_consensus.cc:2243] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.806483 22629 raft_consensus.cc:2272] T d6d3d4f634284532bbf6ad7d4ac19214 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.808835 22629 tablet_replica.cc:333] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.809666 22629 raft_consensus.cc:2243] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.810365 22629 raft_consensus.cc:2272] T 77aa2cbe928a45d4b1652ca10a80aa99 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.813256 22629 tablet_replica.cc:333] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.813884 22629 raft_consensus.cc:2243] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.814563 22629 raft_consensus.cc:2272] T de3c70defd864769b26b52b7f36548cd P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.816865 22629 tablet_replica.cc:333] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.817454 22629 raft_consensus.cc:2243] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.818482 22629 raft_consensus.cc:2272] T 4e60a1c6520545729d27474d8f237ce5 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.821085 22629 tablet_replica.cc:333] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.821678 22629 raft_consensus.cc:2243] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.822495 22629 raft_consensus.cc:2272] T 956a6729b1f34c499670aa77b5896a53 P 05679ef5e2b74861aad2a08b6a05e276 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.824966 22629 tablet_replica.cc:333] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.825559 22629 raft_consensus.cc:2243] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.826483 22629 raft_consensus.cc:2272] T 98e5bc86804f4f979f58779caa696814 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.829061 22629 tablet_replica.cc:333] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.830000 22629 raft_consensus.cc:2243] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.831108 22629 raft_consensus.cc:2272] T 7f5f207bbfed4f119db8464be0a5a8cf P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.833323 22629 tablet_replica.cc:333] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.833937 22629 raft_consensus.cc:2243] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.834934 22629 raft_consensus.cc:2272] T c4369a8cdaaa45beac1359775fa54781 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.837106 22629 tablet_replica.cc:333] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.837700 22629 raft_consensus.cc:2243] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.838517 22629 raft_consensus.cc:2272] T c8b83386d2fa4ed2a212fbe8873239ea P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.840649 22629 tablet_replica.cc:333] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.841297 22629 raft_consensus.cc:2243] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.841773 22629 raft_consensus.cc:2272] T 0c9d6097cb8d40efa55ff97de5b26130 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.843890 22629 tablet_replica.cc:333] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.844648 22629 raft_consensus.cc:2243] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.845571 22629 raft_consensus.cc:2272] T 34d6300a13a3401283a46bf3d2a07e82 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.848219 22629 tablet_replica.cc:333] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.848805 22629 raft_consensus.cc:2243] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.849373 22629 raft_consensus.cc:2272] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.851251 22629 tablet_replica.cc:333] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.851862 22629 raft_consensus.cc:2243] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.852252 22629 raft_consensus.cc:2272] T d9f2f570298740c691fa19d449b3fa4f P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.854290 22629 tablet_replica.cc:333] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.855007 22629 raft_consensus.cc:2243] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.855657 22629 raft_consensus.cc:2272] T 233089f4b82a4b1996ad9129dc392819 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.857985 22629 tablet_replica.cc:333] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.858760 22629 raft_consensus.cc:2243] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.859625 22629 raft_consensus.cc:2272] T 8f44bb7910714c46af4e3d0e2076e7c5 P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.861685 22629 tablet_replica.cc:333] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.862334 22629 raft_consensus.cc:2243] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.862937 22629 raft_consensus.cc:2272] T e1031da3c683416ebfc283d98b45867d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.865226 22629 tablet_replica.cc:333] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.865780 22629 raft_consensus.cc:2243] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.866601 22629 raft_consensus.cc:2272] T 872245a7005f472bb7f6cfafda680b74 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.869014 22629 tablet_replica.cc:333] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.869596 22629 raft_consensus.cc:2243] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.870486 22629 raft_consensus.cc:2272] T 8f333e0a484248ce88f0905e18500838 P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.873000 22629 tablet_replica.cc:333] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.873586 22629 raft_consensus.cc:2243] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.874125 22629 raft_consensus.cc:2272] T 5ed3fc49956b40aa86095410b7c7e5fa P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.876238 22629 tablet_replica.cc:333] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.876813 22629 raft_consensus.cc:2243] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.877462 22629 raft_consensus.cc:2272] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.879674 22629 tablet_replica.cc:333] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.880231 22629 raft_consensus.cc:2243] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.881088 22629 raft_consensus.cc:2272] T 102493ff11494b009f79963bb57be511 P 05679ef5e2b74861aad2a08b6a05e276 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.883162 22629 tablet_replica.cc:333] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.883777 22629 raft_consensus.cc:2243] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.884577 22629 raft_consensus.cc:2272] T 643746a7a11243df94746cf1081fc078 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.886785 22629 tablet_replica.cc:333] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.887392 22629 raft_consensus.cc:2243] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.888571 22629 raft_consensus.cc:2272] T 055432f1640d4d5a8a4ec2355e0b89ac P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.890959 22629 tablet_replica.cc:333] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.891582 22629 raft_consensus.cc:2243] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.892581 22629 raft_consensus.cc:2272] T 3aacc8d80dd2433b94e6d07927304b47 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.894969 22629 tablet_replica.cc:333] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.895540 22629 raft_consensus.cc:2243] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.896109 22629 raft_consensus.cc:2272] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.898370 22629 tablet_replica.cc:333] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.898943 22629 raft_consensus.cc:2243] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.899782 22629 raft_consensus.cc:2272] T a1b618b31b2e427b8c073b2c24771903 P 05679ef5e2b74861aad2a08b6a05e276 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.901906 22629 tablet_replica.cc:333] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.902503 22629 raft_consensus.cc:2243] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.902997 22629 raft_consensus.cc:2272] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.905239 22629 tablet_replica.cc:333] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.906029 22629 raft_consensus.cc:2243] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.906734 22629 raft_consensus.cc:2272] T c395cd7fe30b41869bce6dac359f2cf8 P 05679ef5e2b74861aad2a08b6a05e276 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.909157 22629 tablet_replica.cc:333] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.909762 22629 raft_consensus.cc:2243] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.910616 22629 raft_consensus.cc:2272] T 349b687db5c44196bafc207c26ca3679 P 05679ef5e2b74861aad2a08b6a05e276 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.912628 22629 tablet_replica.cc:333] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.913130 22629 raft_consensus.cc:2243] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.913961 22629 raft_consensus.cc:2272] T 20fc63afa03b45c0bf3f58aae82f3e3d P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.915987 22629 tablet_replica.cc:333] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.916512 22629 raft_consensus.cc:2243] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.917196 22629 raft_consensus.cc:2272] T c7203e0ca92840388d931c03974d32ff P 05679ef5e2b74861aad2a08b6a05e276 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.919235 22629 tablet_replica.cc:333] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.919843 22629 raft_consensus.cc:2243] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.920593 22629 raft_consensus.cc:2272] T 24dc0370e122479081184511cb1224f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.922339 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:17.922745 22629 tablet_replica.cc:333] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.923393 22629 raft_consensus.cc:2243] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.923851 22629 raft_consensus.cc:2272] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.925879 22629 tablet_replica.cc:333] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.926489 22629 raft_consensus.cc:2243] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.926976 22629 raft_consensus.cc:2272] T cc9ef29919aa42aaa2b7545c10e39eff P 05679ef5e2b74861aad2a08b6a05e276 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.929018 22629 tablet_replica.cc:333] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.929569 22629 raft_consensus.cc:2243] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.930485 22629 raft_consensus.cc:2272] T d8341d6ef4764792b1bd6ff196e8af38 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.932717 22629 tablet_replica.cc:333] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.933230 22629 raft_consensus.cc:2243] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.933990 22629 raft_consensus.cc:2272] T ef461146a2924dba8b8e2906d76859f1 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.935809 22629 tablet_replica.cc:333] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.936352 22629 raft_consensus.cc:2243] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:16:17.937055 22629 raft_consensus.cc:2272] T c6f2d1f3ed104c1e99f82ebe04843248 P 05679ef5e2b74861aad2a08b6a05e276 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.939342 22629 tablet_replica.cc:333] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.939893 22629 raft_consensus.cc:2243] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.940377 22629 raft_consensus.cc:2272] T bec362a6d01840d39be46f0c3d28c218 P 05679ef5e2b74861aad2a08b6a05e276 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.942276 22629 tablet_replica.cc:333] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.942764 22629 raft_consensus.cc:2243] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.943192 22629 raft_consensus.cc:2272] T 71cb2ce294574daa8d7244ae252e8aa6 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:17.945258 22629 tablet_replica.cc:333] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: stopping tablet replica
I20260709 12:16:17.945693 22629 raft_consensus.cc:2243] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:17.946143 22629 raft_consensus.cc:2272] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:17.988425 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:17.998592 25113 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.118283 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.134985 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.171043 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.214576 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.297497 25112 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.302017 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.308488 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.329800 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.401816 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.456813 25119 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.545332 25114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:18.552289 30772 leader_election.cc:336] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:18.552843 30772 leader_election.cc:304] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:18.553544 30772 raft_consensus.cc:2686] T 526cc198da134bb590358ed73c23d59e P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:18.555449 22629 leader_election.cc:336] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Aborted: reactor is shutting down (error 108)
W20260709 12:16:18.555800 30772 leader_election.cc:336] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:18.555938 22629 leader_election.cc:304] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 0ee4ab0747914e7abc642312517b5e25, 438159d8de3c4027bdcf57eb81dedf86
I20260709 12:16:18.556241 30772 leader_election.cc:304] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:18.556641 22629 raft_consensus.cc:2686] T d79b6f8bf5274da7a3f667b9988c5a84 P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:18.556763 30772 raft_consensus.cc:2686] T 1567865ea02a4884996ae9d8beb9fa29 P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:18.558652 22629 leader_election.cc:336] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Aborted: reactor is shutting down (error 108)
W20260709 12:16:18.559111 30772 leader_election.cc:336] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:18.559077 22629 leader_election.cc:304] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
I20260709 12:16:18.559633 30772 leader_election.cc:304] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:18.559870 22629 raft_consensus.cc:2686] T 78f1c4dc751a4f4a93a446a785edda51 P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:18.560159 30772 raft_consensus.cc:2686] T 10c8a875cf4b4acea62c1086994499e4 P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:18.561911 30772 leader_election.cc:336] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 438159d8de3c4027bdcf57eb81dedf86 (127.22.25.66:44897): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:18.562265 30772 leader_election.cc:304] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05679ef5e2b74861aad2a08b6a05e276; no voters: 438159d8de3c4027bdcf57eb81dedf86, 9e0bf56d58c5480a8d088c85566558da
W20260709 12:16:18.562789 30772 raft_consensus.cc:2686] T 8368f2e695204683affed6a9bd84ba96 P 05679ef5e2b74861aad2a08b6a05e276: unable to run election callback: Service unavailable: The pool has been shut down.
F20260709 12:16:18.564877 22629 leader_election.cc:206] Check failed: has_responded_ 
*** Check failure stack trace: ***
*** Aborted at 1783599378 (unix time) try "date -d @1783599378" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e800005865) received by PID 22629 (TID 0x7fb2a62f8840) from PID 22629; stack trace: ***
    @           0x2d4280 __tsan::CallUserSignalHandler() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
    @           0x2d6763 rtl_sigaction() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
    @     0x7fb2adbba980 (unknown) at ??:0
    @     0x7fb2a6fb8e87 gsignal at ??:0
    @     0x7fb2a6fba7f1 abort at ??:0
    @           0x2da276 abort at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
    @     0x7fb2a9d8ad84 google::LogMessage::SendToLog() at ??:0
    @     0x7fb2a9d8b930 google::LogMessage::Flush() at ??:0
    @     0x7fb2a9d90a6b google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7fb2ad75ec9c kudu::consensus::LeaderElection::~LeaderElection() at ??:0
    @     0x7fb2ad76c4df kudu::RefCountedThreadSafe<>::DeleteInternal() at ??:0
    @     0x7fb2ad76c4aa kudu::DefaultRefCountedThreadSafeTraits<>::Destruct() at ??:0
    @     0x7fb2ad76c466 kudu::RefCountedThreadSafe<>::Release() at ??:0
    @     0x7fb2ad763f37 scoped_refptr<>::~scoped_refptr() at ??:0
    @     0x7fb2ad7602d3 kudu::consensus::LeaderElection::Run()::$_0::~Run() at ??:0
    @     0x7fb2ad761eba std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7fb2ad7627da std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7fb2ad761aab std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x376a3b std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x36234a std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
    @     0x7fb2aba8223e kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest()::$_0::~RefreshDnsAndEnqueueRequest() at ??:0
    @     0x7fb2aba835fa std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7fb2aba83f2a std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7fb2aba831cb std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @     0x7fb2b35c831b std::__1::__function::__value_func<>::~__value_func() at ??:0
    @     0x7fb2b35993ba std::__1::function<>::~function() at ??:0
    @     0x7fb2aa640f6b kudu::DnsResolver::RefreshAddressesAsync()::$_1::~RefreshAddressesAsync() at ??:0
    @     0x7fb2aa64316a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7fb2aa643ada std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7fb2aa642d0b std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x376a3b std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x36234a std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
